-- Logs begin at Wed 2022-08-24 05:47:43 UTC, end at Wed 2022-08-24 07:04:29 UTC. -- Aug 24 06:20:21.149140 dv-cinder-854270-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 24 06:20:21.158104 dv-cinder-854270-1-iscsi devstack@keystone.service[81714]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 24 06:20:21.160298 dv-cinder-854270-1-iscsi devstack@keystone.service[81714]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Aug 24 06:20:21 2022] *** Aug 24 06:20:21.160298 dv-cinder-854270-1-iscsi devstack@keystone.service[81714]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 24 06:20:21.160298 dv-cinder-854270-1-iscsi devstack@keystone.service[81714]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 24 06:20:21.160298 dv-cinder-854270-1-iscsi devstack@keystone.service[81714]: nodename: dv-cinder-854270-1-iscsi Aug 24 06:20:21.160298 dv-cinder-854270-1-iscsi devstack@keystone.service[81714]: machine: x86_64 Aug 24 06:20:21.160298 dv-cinder-854270-1-iscsi devstack@keystone.service[81714]: clock source: unix Aug 24 06:20:21.160298 dv-cinder-854270-1-iscsi devstack@keystone.service[81714]: pcre jit disabled Aug 24 06:20:21.160298 dv-cinder-854270-1-iscsi devstack@keystone.service[81714]: detected number of CPU cores: 4 Aug 24 06:20:21.160298 dv-cinder-854270-1-iscsi devstack@keystone.service[81714]: current working directory: / Aug 24 06:20:21.160298 dv-cinder-854270-1-iscsi devstack@keystone.service[81714]: detected binary path: /usr/bin/uwsgi-core Aug 24 06:20:21.160298 dv-cinder-854270-1-iscsi devstack@keystone.service[81714]: your processes number limit is 47680 Aug 24 06:20:21.160909 dv-cinder-854270-1-iscsi devstack@keystone.service[81714]: your memory page size is 4096 bytes Aug 24 06:20:21.160909 dv-cinder-854270-1-iscsi devstack@keystone.service[81714]: detected max file descriptor number: 2048 Aug 24 06:20:21.160909 dv-cinder-854270-1-iscsi devstack@keystone.service[81714]: lock engine: pthread robust mutexes Aug 24 06:20:21.160909 dv-cinder-854270-1-iscsi devstack@keystone.service[81714]: thunder lock: enabled Aug 24 06:20:21.160909 dv-cinder-854270-1-iscsi devstack@keystone.service[81714]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 24 06:20:21.160909 dv-cinder-854270-1-iscsi devstack@keystone.service[81714]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 24 06:20:21.199482 dv-cinder-854270-1-iscsi devstack@keystone.service[81714]: Python main interpreter initialized at 0x564396d869c0 Aug 24 06:20:21.199482 dv-cinder-854270-1-iscsi devstack@keystone.service[81714]: python threads support enabled Aug 24 06:20:21.199482 dv-cinder-854270-1-iscsi devstack@keystone.service[81714]: your server socket listen backlog is limited to 100 connections Aug 24 06:20:21.199482 dv-cinder-854270-1-iscsi devstack@keystone.service[81714]: your mercy for graceful operations on workers is 90 seconds Aug 24 06:20:21.199862 dv-cinder-854270-1-iscsi devstack@keystone.service[81714]: mapped 403077 bytes (393 KB) for 2 cores Aug 24 06:20:21.199939 dv-cinder-854270-1-iscsi devstack@keystone.service[81714]: *** Operational MODE: preforking *** Aug 24 06:20:21.200184 dv-cinder-854270-1-iscsi devstack@keystone.service[81714]: *** uWSGI is running in multiple interpreter mode *** Aug 24 06:20:21.200184 dv-cinder-854270-1-iscsi devstack@keystone.service[81714]: spawned uWSGI master process (pid: 81714) Aug 24 06:20:21.200394 dv-cinder-854270-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 24 06:20:21.201288 dv-cinder-854270-1-iscsi devstack@keystone.service[81714]: spawned uWSGI worker 1 (pid: 81723, cores: 1) Aug 24 06:20:21.201288 dv-cinder-854270-1-iscsi devstack@keystone.service[81714]: spawned uWSGI worker 2 (pid: 81724, cores: 1) Aug 24 06:20:21.201450 dv-cinder-854270-1-iscsi devstack@keystone.service[81714]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 24 06:20:25.688095 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 06:20:25.688095 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: from cryptography.utils import int_from_bytes Aug 24 06:20:25.688868 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 06:20:25.688868 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: from cryptography.utils import int_from_bytes Aug 24 06:20:25.690152 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 06:20:25.690152 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: from cryptography.utils import int_from_bytes Aug 24 06:20:25.690623 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 06:20:25.690623 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: from cryptography.utils import int_from_bytes Aug 24 06:20:25.746530 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 24 06:20:25.746723 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 24 06:20:25.746813 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 24 06:20:25.746901 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 24 06:20:25.747030 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] command line args: [] {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 24 06:20:25.747156 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] command line args: [] {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 24 06:20:25.747302 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 24 06:20:25.747394 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 24 06:20:25.747471 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ================================================================================ {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 24 06:20:25.747557 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] ================================================================================ {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 24 06:20:25.747726 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] admin_token = **** {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.747809 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] admin_token = **** {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.747971 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] config_dir = [] {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.748057 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] config_dir = [] {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.748157 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.748247 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.748326 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] config_source = [] {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.748402 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] config_source = [] {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.748498 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] debug = True {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.748576 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] debug = True {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.748694 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.748951 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.749044 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.749105 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.749281 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.749378 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.749483 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.749562 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.749727 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.749814 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.749956 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] list_limit = None {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.750047 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] list_limit = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.750151 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] log_config_append = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.750229 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] log_config_append = None {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.750295 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.750358 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.750469 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] log_dir = None {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.750553 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] log_dir = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.750659 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] log_file = None {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.750733 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] log_file = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.750873 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.750959 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.751118 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.751221 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.751452 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.751541 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.751723 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.751815 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.751886 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.751952 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.752789 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.752908 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.752908 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.753037 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.753109 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.753175 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.753388 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.753485 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.753571 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.753644 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.753772 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.753880 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.754004 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.754094 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.754253 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.754328 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.754502 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.754581 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.754781 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.754878 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.755284 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.47/identity {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.755384 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.47/identity {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.755458 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] publish_errors = False {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.755586 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] publish_errors = False {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.755682 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.755760 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.755909 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.755988 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.756102 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.756211 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.756332 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.756431 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.756566 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.756653 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.756799 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] standard_threads = False {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.756886 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] standard_threads = False {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.757027 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.757140 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.757291 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.763628 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.763744 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.763823 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.763895 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] use_journal = False {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.763972 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] use_journal = False {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.764049 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] use_json = False {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.764121 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] use_json = False {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.764199 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] use_stderr = False {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.764277 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] use_stderr = False {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.764354 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] use_syslog = False {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.764426 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] use_syslog = False {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.764501 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.764578 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 06:20:25.764729 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.764805 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.764875 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.764946 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.765022 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.765099 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.765201 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.765288 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.765367 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.765440 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.765518 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.765599 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.765680 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.765757 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.765829 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.765829 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.765956 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.766026 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.766103 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.766174 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.766244 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.766314 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.766383 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.766453 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.766531 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.766603 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.766673 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.766743 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.766813 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.766882 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.766964 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.767042 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.767119 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.767191 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.767260 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.767337 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.767409 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.767479 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.767554 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.767626 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.767695 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.767803 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.767887 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.768019 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.768099 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.768171 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.768241 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.768387 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.768462 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.768533 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.768617 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.768698 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.768781 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.768862 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.768986 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.769062 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.769140 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.769243 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.769316 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.769403 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.769482 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.769559 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.769631 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.769701 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.769771 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.769848 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.769920 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.769996 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.770068 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.770143 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.770220 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.770299 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.770377 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.770449 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.770518 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.770623 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.770720 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.770824 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.770922 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.771016 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.771111 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.771236 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.771345 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.771452 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.771452 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.771632 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.771729 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.771809 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.771927 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.771927 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.772117 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.772192 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.772269 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.772341 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.772466 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.772553 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.772628 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.772752 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.772870 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.772946 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.773062 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.773138 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.773283 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.773362 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.773479 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.773555 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.773671 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.773747 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.773828 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.773901 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.774026 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.774104 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.774181 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.774254 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.774333 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.774411 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.774484 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.774555 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.774627 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.774705 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.774783 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.774879 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.774949 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.775029 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.775091 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.775161 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.775236 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.775316 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.775387 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.775461 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.775535 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.775608 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.775687 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.775765 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.775837 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.775903 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.775972 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.776042 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.776124 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.776201 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.776201 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.776319 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.776388 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.776458 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.776607 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.776683 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.776754 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.776825 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.776895 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.776961 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.777044 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.777117 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.777217 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.777293 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.777364 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.777434 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.777521 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.777609 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.777687 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.777760 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.777830 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.777901 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.777971 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.778049 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.778123 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.778194 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.778359 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.778447 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.778599 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.778690 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.778838 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.778920 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.779080 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.779161 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.779318 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.779406 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.779553 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.779633 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.779807 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.779954 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.780144 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.780496 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.780579 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.780653 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.780761 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.780841 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.780996 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.781084 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.781246 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.781330 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.781486 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.781567 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.781721 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.781811 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.781966 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.782052 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.782201 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.782282 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.782442 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.782532 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.782683 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.782785 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.782934 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.783014 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.783187 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.783286 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] auth.external = None {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.784010 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.784099 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.784309 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.784389 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.784549 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.784653 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.784781 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] auth.password = None {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.784904 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] auth.external = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.785014 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] auth.token = None {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.785096 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.785276 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.792191 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.792311 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.792409 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.792487 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.792560 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] auth.password = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.792632 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.792703 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] auth.token = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.792774 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.792854 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.792927 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.793004 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.793084 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.793156 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.793255 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] credential.caching = True {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.793337 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.793409 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.793480 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.793550 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.793620 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.793697 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.793776 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.793849 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.793919 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] credential.caching = True {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.793989 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.794059 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.794129 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.794203 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.794276 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.794346 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.794417 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.794486 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.794563 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.794635 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.794706 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.794775 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.794845 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.794915 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.794994 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.795066 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.795136 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.795206 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.795277 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.795346 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.795494 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.795569 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.795648 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] federation.caching = True {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.795721 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.795791 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.795862 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.795940 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.796017 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.796089 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.796159 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.796229 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.796299 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] federation.caching = True {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.796377 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.796449 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.796519 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.796519 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.796645 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.796721 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.796800 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.796872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.796942 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.797047 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.797135 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.797246 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.797323 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.797398 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] identity.caching = True {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.797469 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.797540 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.797616 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.797688 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.797766 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.797838 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.797914 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.797982 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.798053 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.798131 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.798209 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] identity.caching = True {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.798281 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.798351 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.798421 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.798491 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.798560 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.798638 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.798710 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.798785 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.798856 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.798927 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.799045 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.799645 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.799723 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.799795 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.799866 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.799937 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.799996 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.800062 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.800133 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.800337 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.800419 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.800491 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.800561 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.800640 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.800712 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.800783 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.800853 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.800853 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.800979 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.800979 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.801115 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.801202 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.801276 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.801346 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.801429 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.801429 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.801559 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.801559 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.801693 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.801763 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.801858 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.801932 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.802003 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.802073 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.802142 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.802212 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.802290 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.802362 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.802447 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.802541 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.802615 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.802676 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.802741 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.802813 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.802884 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.802953 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.803023 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.803100 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.803179 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.803251 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.803321 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.803391 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.803460 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.803529 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.803606 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.803678 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.803748 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.803817 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.803886 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.803955 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.804032 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.804104 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.804174 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.804244 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.804315 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.804385 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.804462 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.804462 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.804591 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.804717 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.804800 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.804963 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.805052 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.805216 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.805323 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.805497 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.805583 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.805748 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.805842 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.805909 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] ldap.user = None {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.805969 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.806212 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.806293 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.806412 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.806521 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.806655 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.806756 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.806893 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.807003 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.807131 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.807252 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.807364 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.user = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.807498 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.807609 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.807733 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.807914 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.808010 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.808147 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.808234 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.808395 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.808482 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.808628 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.808715 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.808858 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.808937 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.809103 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.809198 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.809357 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.809457 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.809595 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.809674 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.809832 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.809924 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.810072 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.810161 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.810300 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.810379 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.810545 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.810624 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.810781 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.810868 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.811017 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.811097 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.811253 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.811332 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.811489 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.811576 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.811723 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.811801 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.811967 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.812044 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.812203 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.812291 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.812440 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.812518 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.812674 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.812752 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.812909 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.812988 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.813142 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.813267 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.813401 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.813480 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.813631 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.813708 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.813870 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.813951 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.814103 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.814179 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.814325 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.814561 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.814677 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.815755 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] resource.caching = True {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.815936 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.816013 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.816096 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.816170 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.816241 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.816313 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.816383 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.816454 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.816532 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.816606 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.816684 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.816756 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.816827 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.816898 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.816977 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] resource.caching = True {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.817049 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.817120 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.817212 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.817287 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.817359 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] role.caching = True {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.817443 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.817517 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] role.driver = sql {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.817588 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.817660 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.817731 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.817851 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.817940 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.818099 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.818180 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.818346 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.818425 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.818590 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.818676 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.818831 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] role.caching = True {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.818910 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.819071 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.819151 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] role.driver = sql {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.819316 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.819404 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.819561 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.819640 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.819792 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.819871 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.820037 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.820125 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.820280 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.820361 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.820522 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.820602 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.820766 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.820853 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.821012 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.821249 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.821334 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.821444 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.821611 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.830683 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.830816 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.830920 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.830995 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.831066 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.831149 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.831224 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.831295 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.831366 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.831366 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.831494 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.831572 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.831712 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.831831 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.831907 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.831978 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.832049 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.832128 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.832191 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.832261 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.832331 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.832401 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.832470 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.832548 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.832621 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.832692 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.832761 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.832832 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.832901 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.832978 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.833050 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.833120 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.833218 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.833293 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] token.caching = True {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.833364 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.833447 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.833520 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.833590 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.833660 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.833730 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.833800 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.833879 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.833951 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.834021 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.834091 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.834161 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.834231 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.834309 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.834382 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] token.caching = True {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.834452 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.834523 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.834593 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.834663 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.834741 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.834814 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.834884 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.834884 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.835010 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.835080 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.835148 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.835219 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.835290 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.835360 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.835430 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.835500 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.835578 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.835578 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.835706 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.835783 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.835854 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] database.connection = **** {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.835932 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.836010 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.836090 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.836161 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.836232 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.836302 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.836302 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.836438 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.836516 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.836588 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.836659 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.836729 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.836816 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.836816 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.836947 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.837008 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.837078 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] database.connection = **** {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.837157 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.837291 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.837366 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.837443 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.837588 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.837664 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.837734 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.837804 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.837876 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.837946 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.838024 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.838097 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.838168 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.838239 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.838309 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.838378 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.838461 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.838534 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.838604 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81723) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 24 06:20:25.838675 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.838675 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.838675 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.838675 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.838675 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.838675 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.838675 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.838675 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.838675 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 06:20:25.839202 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81724) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 24 06:20:25.864360 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81723) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:20:25.864584 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81723) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:20:25.864808 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81723) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:20:25.866421 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81724) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:20:25.866657 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81724) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:20:25.866886 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81724) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:20:25.873246 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81723) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:20:25.875327 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81724) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:20:25.902795 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81723) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:20:25.903018 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81723) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:20:25.910884 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81724) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:20:25.911117 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81724) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:20:25.966589 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81723) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:20:25.967154 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81724) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:20:26.028859 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81724) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:20:26.029217 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81724) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:20:26.036087 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81723) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:20:26.036315 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81723) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:20:26.192022 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81723) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:20:26.192292 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81723) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:20:26.192445 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81723) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:20:26.192575 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81723) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:20:26.192792 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81723) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:20:26.193010 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81723) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:20:26.197465 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81724) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:20:26.197696 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81724) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:20:26.197921 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81724) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:20:26.198142 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81724) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:20:26.198361 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81724) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:20:26.198579 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81724) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:20:26.221460 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81723) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:20:26.221698 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81723) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:20:26.221783 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81724) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:20:26.221909 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81723) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:20:26.221994 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81724) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:20:26.222132 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81723) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:20:26.222346 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81724) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:20:26.222434 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81723) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:20:26.222509 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81724) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:20:26.222581 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81723) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:20:26.222653 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81724) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:20:26.222763 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81723) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:20:26.222848 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81724) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:20:26.223155 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81724) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:20:26.234080 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81723) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:20:26.234514 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81724) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:20:26.247422 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81723) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:20:26.254862 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81723) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:20:26.254862 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81723) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:20:26.254862 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81723) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:20:26.255110 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81724) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:20:26.255190 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81723) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:20:26.255282 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81724) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:20:26.255362 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81723) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:20:26.255425 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81724) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:20:26.255485 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81723) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:20:26.255545 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81724) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:20:26.255545 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81724) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:20:26.255545 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81724) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:20:26.255545 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81724) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 06:20:26.271799 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.272017 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 06:20:26.272351 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.272553 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 06:20:26.272828 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.273089 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.273659 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.273917 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.274017 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.274128 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 06:20:26.274473 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.274676 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 06:20:26.274949 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.275337 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.275860 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.276121 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.284384 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.284660 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.284956 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.285255 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.286848 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.287127 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.287395 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.287675 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.292346 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81724) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 06:20:26.294645 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81723) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 06:20:26.295319 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81724) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 06:20:26.295599 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.295882 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.296166 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.296452 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.296710 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.296978 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.297278 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.297538 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81723) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 06:20:26.297666 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.297824 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.297945 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.298108 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.298218 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.298392 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.298674 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.298934 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.299234 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.299506 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.299890 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.300212 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.300532 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.322563 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.323223 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.324051 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81724) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 06:20:26.324337 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.324694 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81723) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 06:20:26.324979 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.329274 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81724) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 06:20:26.329734 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81723) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 06:20:26.329847 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.329936 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.330081 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.330739 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.337073 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81723) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 06:20:26.337383 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.337482 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81724) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 06:20:26.337755 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.341476 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81723) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 06:20:26.341769 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.341851 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81724) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 06:20:26.342051 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.342136 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.342333 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.342415 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.342647 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.342752 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.342885 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.342970 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.343156 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.343239 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.343424 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.343516 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.343691 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.361669 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.361943 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.362309 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.368640 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.368788 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81723) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 06:20:26.368886 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81724) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 06:20:26.368994 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.369162 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.369948 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.370246 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.373332 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81723) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 06:20:26.374483 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81724) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 06:20:26.376482 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81723) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 06:20:26.378292 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81724) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 06:20:26.380516 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.380842 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.381119 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.381427 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.381696 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.381807 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.381995 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.382087 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.382286 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.382582 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.382872 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.383154 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.398927 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81723) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 06:20:26.399244 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.399513 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.399776 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.400633 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81724) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 06:20:26.400944 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.401250 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.401518 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.406977 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.408679 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.409014 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.409168 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.410691 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.410966 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.411728 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81723) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 06:20:26.412021 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.412314 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.412605 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.412885 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.414030 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81724) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 06:20:26.414326 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.414615 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.414916 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.420289 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.424872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81723) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 06:20:26.425199 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.425486 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.425766 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.426046 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.426321 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.426597 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.427268 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81724) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 06:20:26.427576 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.427859 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.428141 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.428421 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.428696 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.428972 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.442929 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81723) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 06:20:26.445907 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81724) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 06:20:26.446047 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81723) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 06:20:26.448979 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.449082 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81724) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 06:20:26.450466 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.451943 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81723) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 06:20:26.452193 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.452279 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.452507 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.453663 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.455283 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81724) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 06:20:26.456067 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.456366 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.459928 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.461447 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81723) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 06:20:26.463810 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.464339 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.464650 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.464937 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.465246 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.465539 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81724) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 06:20:26.468586 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.468918 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.469235 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.469519 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.473371 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81723) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 06:20:26.477671 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.477671 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.478478 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81724) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 06:20:26.478788 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.479072 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.481000 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81723) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 06:20:26.481320 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.481603 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.481879 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.482153 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.482427 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.482700 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.482983 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.483256 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.483527 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.483796 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.484069 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.484340 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.484606 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81723) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.486197 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81724) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 06:20:26.486517 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.486860 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.487072 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.487346 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.487633 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.487905 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.488218 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.488563 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.488835 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.489115 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.489401 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.489681 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.489950 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81724) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 06:20:26.520327 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x564396d869c0 pid: 81723 (default app) Aug 24 06:20:26.523939 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9badcc26-ceee-4044-bc1d-c5d4aaad821b None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:26.524229 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9badcc26-ceee-4044-bc1d-c5d4aaad821b None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:26.524499 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9badcc26-ceee-4044-bc1d-c5d4aaad821b None None] PATH_INFO: `/v3` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:26.525927 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x564396d869c0 pid: 81724 (default app) Aug 24 06:20:26.528762 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1/1] 10.222.0.47 () {50 vars in 791 bytes} [Wed Aug 24 06:20:26 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:20:34.208583 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf024435-531f-48c7-921b-e779d2245855 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:34.209154 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf024435-531f-48c7-921b-e779d2245855 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:34.209154 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf024435-531f-48c7-921b-e779d2245855 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:34.213338 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1/2] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:20:34 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:20:34.220212 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28f257f-e62f-4ff0-9836-5727e03ad622 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:34.221266 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28f257f-e62f-4ff0-9836-5727e03ad622 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:34.221490 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28f257f-e62f-4ff0-9836-5727e03ad622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:34.294114 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: INFO dbcounter [None req-e28f257f-e62f-4ff0-9836-5727e03ad622 None None] Registered counter for database keystone Aug 24 06:20:34.315889 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG oslo_db.sqlalchemy.engines [None req-e28f257f-e62f-4ff0-9836-5727e03ad622 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81723) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 06:20:34.322560 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writer thread running {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 06:20:34.514432 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG passlib.handlers.bcrypt [None req-e28f257f-e62f-4ff0-9836-5727e03ad622 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81723) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 24 06:20:34.514773 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG passlib.handlers.bcrypt [None req-e28f257f-e62f-4ff0-9836-5727e03ad622 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81723) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 24 06:20:34.571952 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-e28f257f-e62f-4ff0-9836-5727e03ad622 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:20:34.573615 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-e28f257f-e62f-4ff0-9836-5727e03ad622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:20:34.707477 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 2/3] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:20:34 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 488 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 24 06:20:35.179806 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf024435-531f-48c7-921b-e779d2245855 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:20:35.249908 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: INFO dbcounter [None req-cf024435-531f-48c7-921b-e779d2245855 None None] Registered counter for database keystone Aug 24 06:20:35.271259 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG oslo_db.sqlalchemy.engines [None req-cf024435-531f-48c7-921b-e779d2245855 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81724) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 06:20:35.277541 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writer thread running {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 06:20:35.404088 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf024435-531f-48c7-921b-e779d2245855 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:20:35.409480 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accb9548-8cc0-42ea-a711-00d1409b08a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:20:35.410106 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accb9548-8cc0-42ea-a711-00d1409b08a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:35.410387 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accb9548-8cc0-42ea-a711-00d1409b08a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:35.410657 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accb9548-8cc0-42ea-a711-00d1409b08a3 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:35.419254 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accb9548-8cc0-42ea-a711-00d1409b08a3 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:20:35.475016 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: WARNING keystone.common.rbac_enforcer.enforcer [None req-accb9548-8cc0-42ea-a711-00d1409b08a3 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 24 06:20:35.475255 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accb9548-8cc0-42ea-a711-00d1409b08a3 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:20:35.485548 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 2/4] 10.222.0.47 () {58 vars in 1324 bytes} [Wed Aug 24 06:20:35 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 06:20:35.492385 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28f257f-e62f-4ff0-9836-5727e03ad622 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:20:35.510775 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28f257f-e62f-4ff0-9836-5727e03ad622 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:20:35.515388 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee6ac7e-f56f-4e78-a75f-974edaae22a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:20:35.516548 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee6ac7e-f56f-4e78-a75f-974edaae22a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:35.516837 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee6ac7e-f56f-4e78-a75f-974edaae22a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:35.517115 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee6ac7e-f56f-4e78-a75f-974edaae22a9 admin admin] PATH_INFO: `/v3/projects/24c1e9e70e07496595a602ca047e6a1f` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:35.519112 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee6ac7e-f56f-4e78-a75f-974edaae22a9 admin admin] RBAC: Authorizing `identity:get_project(project_id=24c1e9e70e07496595a602ca047e6a1f)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:20:35.573299 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: WARNING keystone.common.rbac_enforcer.enforcer [None req-bee6ac7e-f56f-4e78-a75f-974edaae22a9 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 24 06:20:35.573646 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee6ac7e-f56f-4e78-a75f-974edaae22a9 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:20:35.578367 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 3/5] 10.222.0.47 () {56 vars in 1207 bytes} [Wed Aug 24 06:20:35 2022] GET /identity/v3/projects/24c1e9e70e07496595a602ca047e6a1f => generated 344 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:20:37.485271 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089a7744-0ff1-4c25-b4cc-51a87d201862 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:37.485789 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089a7744-0ff1-4c25-b4cc-51a87d201862 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:37.485893 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089a7744-0ff1-4c25-b4cc-51a87d201862 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:37.490274 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 3/6] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:20:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 06:20:37.496732 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc7e15c-37b2-4d1c-95a4-7a5d31a35587 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:37.497215 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc7e15c-37b2-4d1c-95a4-7a5d31a35587 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:37.497305 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc7e15c-37b2-4d1c-95a4-7a5d31a35587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:37.519198 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-0fc7e15c-37b2-4d1c-95a4-7a5d31a35587 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:20:37.520727 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-0fc7e15c-37b2-4d1c-95a4-7a5d31a35587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:20:37.528726 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 4/7] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:20:37 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 24 06:20:37.799552 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089a7744-0ff1-4c25-b4cc-51a87d201862 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:20:37.811628 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089a7744-0ff1-4c25-b4cc-51a87d201862 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:20:37.816933 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aebff6-7652-4b7d-8988-9161f1243bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:20:37.817575 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aebff6-7652-4b7d-8988-9161f1243bb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:37.817859 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aebff6-7652-4b7d-8988-9161f1243bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:37.818134 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aebff6-7652-4b7d-8988-9161f1243bb8 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:37.826229 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8aebff6-7652-4b7d-8988-9161f1243bb8 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:20:37.829670 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8aebff6-7652-4b7d-8988-9161f1243bb8 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:20:37.838274 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 4/8] 10.222.0.47 () {58 vars in 1324 bytes} [Wed Aug 24 06:20:37 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 06:20:37.843997 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc7e15c-37b2-4d1c-95a4-7a5d31a35587 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:20:37.853702 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc7e15c-37b2-4d1c-95a4-7a5d31a35587 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:20:37.858132 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fa4bf6-0fc3-4b5b-a4c3-e90596a37b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:20:37.858771 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fa4bf6-0fc3-4b5b-a4c3-e90596a37b3f admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:37.859055 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fa4bf6-0fc3-4b5b-a4c3-e90596a37b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:37.859330 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fa4bf6-0fc3-4b5b-a4c3-e90596a37b3f admin admin] PATH_INFO: `/v3/users/029600e45e6b473db6751098bdba45e2` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:37.860659 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fa4bf6-0fc3-4b5b-a4c3-e90596a37b3f admin admin] RBAC: Authorizing `identity:get_user(user_id=029600e45e6b473db6751098bdba45e2)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:20:37.862844 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fa4bf6-0fc3-4b5b-a4c3-e90596a37b3f admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:20:37.864400 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 5/9] 10.222.0.47 () {56 vars in 1198 bytes} [Wed Aug 24 06:20:37 2022] GET /identity/v3/users/029600e45e6b473db6751098bdba45e2 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:20:39.873512 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29e90fc-8ab8-4cbf-aa5c-3fa5c7c5aeda None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:39.874013 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29e90fc-8ab8-4cbf-aa5c-3fa5c7c5aeda None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:39.874013 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29e90fc-8ab8-4cbf-aa5c-3fa5c7c5aeda None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:39.875295 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 5/10] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:20:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:20:39.881089 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db32fdd1-0335-4283-98de-c6b31de769fa None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:39.881557 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db32fdd1-0335-4283-98de-c6b31de769fa None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:39.881799 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db32fdd1-0335-4283-98de-c6b31de769fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:39.901001 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-db32fdd1-0335-4283-98de-c6b31de769fa None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:20:39.902211 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-db32fdd1-0335-4283-98de-c6b31de769fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:20:39.947458 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3432912-edf7-4577-b3a3-b7c7825dfd38 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:39.947765 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3432912-edf7-4577-b3a3-b7c7825dfd38 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:39.948043 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3432912-edf7-4577-b3a3-b7c7825dfd38 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:39.949393 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 6/11] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:20:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 06:20:39.955535 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8c491d-ddd3-4286-aff4-923782fba3f7 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:39.955960 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8c491d-ddd3-4286-aff4-923782fba3f7 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:39.956305 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8c491d-ddd3-4286-aff4-923782fba3f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:39.988221 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 6/12] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:20:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:20:39.998381 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG passlib.handlers.bcrypt [None req-1e8c491d-ddd3-4286-aff4-923782fba3f7 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81724) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 24 06:20:39.998808 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG passlib.handlers.bcrypt [None req-1e8c491d-ddd3-4286-aff4-923782fba3f7 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81724) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 24 06:20:40.046863 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-1e8c491d-ddd3-4286-aff4-923782fba3f7 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:20:40.048619 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-1e8c491d-ddd3-4286-aff4-923782fba3f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:20:40.152711 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 7/13] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:20:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:20:40.271628 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db32fdd1-0335-4283-98de-c6b31de769fa None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:20:40.296775 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:20:40.379962 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db32fdd1-0335-4283-98de-c6b31de769fa None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:20:40.413990 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d17a7a-2b1b-4add-8d28-e463e86f34dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:20:40.414568 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d17a7a-2b1b-4add-8d28-e463e86f34dd None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:40.414852 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d17a7a-2b1b-4add-8d28-e463e86f34dd None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:40.415131 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d17a7a-2b1b-4add-8d28-e463e86f34dd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:40.423605 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8c491d-ddd3-4286-aff4-923782fba3f7 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:20:40.427361 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d17a7a-2b1b-4add-8d28-e463e86f34dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:20:40.436265 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-32d17a7a-2b1b-4add-8d28-e463e86f34dd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:20:40.436265 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:20:40.436265 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:20:40.436265 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:20:40.436265 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:20:40.436265 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:20:40.436265 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:20:40.436265 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:20:40.436265 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:20:40.436265 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:20:40.436265 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:20:40.436265 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:20:40.436265 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:20:40.436265 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:20:40.436265 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:20:40.436265 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:20:40.437150 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:20:40.437150 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:20:40.437150 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:20:40.437150 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:20:40.437150 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:20:40.437150 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:20:40.437150 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:20:40.437150 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:20:40.437150 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:20:40.437150 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:20:40.437150 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:20:40.437150 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:20:40.437150 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:20:40.437150 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:20:40.437150 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:20:40.437150 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:20:40.442064 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:20:40.442064 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:20:40.442064 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:20:40.442064 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:20:40.442064 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:20:40.442064 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:20:40.442064 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:20:40.442064 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:20:40.442064 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:20:40.442064 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:20:40.442064 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:20:40.442064 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:20:40.442064 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 7/14] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Aug 24 06:20:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:20:40.443921 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d17a7a-2b1b-4add-8d28-e463e86f34dd None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:20:40.532310 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:20:40.534452 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8c491d-ddd3-4286-aff4-923782fba3f7 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:20:40.536854 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:20:40.551267 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d17a7a-2b1b-4add-8d28-e463e86f34dd None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:20:40.572442 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6211fad0-1f0b-4e33-8df4-c72c543018a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:20:40.573337 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6211fad0-1f0b-4e33-8df4-c72c543018a6 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:40.573625 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6211fad0-1f0b-4e33-8df4-c72c543018a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:40.573902 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6211fad0-1f0b-4e33-8df4-c72c543018a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:40.588298 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf47bb1-6faa-419e-9e7f-c4fcfea11af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:20:40.588955 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf47bb1-6faa-419e-9e7f-c4fcfea11af3 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:40.589269 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf47bb1-6faa-419e-9e7f-c4fcfea11af3 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:40.589547 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf47bb1-6faa-419e-9e7f-c4fcfea11af3 None admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:40.590237 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf47bb1-6faa-419e-9e7f-c4fcfea11af3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:20:40.590549 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf47bb1-6faa-419e-9e7f-c4fcfea11af3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:20:40.592007 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf47bb1-6faa-419e-9e7f-c4fcfea11af3 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:20:40.605910 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 8/15] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 24 06:20:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:20:40.612877 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf47bb1-6faa-419e-9e7f-c4fcfea11af3 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:20:40.649682 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6211fad0-1f0b-4e33-8df4-c72c543018a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:20:40.651309 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6211fad0-1f0b-4e33-8df4-c72c543018a6 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:20:40.716772 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf47bb1-6faa-419e-9e7f-c4fcfea11af3 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:20:40.750077 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 8/16] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 24 06:20:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 06:20:40.752913 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e32c6c1-1e2f-43b7-9c82-8e9b9748ecba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:20:40.753592 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e32c6c1-1e2f-43b7-9c82-8e9b9748ecba None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:40.753877 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e32c6c1-1e2f-43b7-9c82-8e9b9748ecba None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:40.754150 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e32c6c1-1e2f-43b7-9c82-8e9b9748ecba None admin] PATH_INFO: `/v3/users/029600e45e6b473db6751098bdba45e2` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:40.755397 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e32c6c1-1e2f-43b7-9c82-8e9b9748ecba None admin] RBAC: Authorizing `identity:get_user(user_id=029600e45e6b473db6751098bdba45e2)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:20:40.755612 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6211fad0-1f0b-4e33-8df4-c72c543018a6 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:20:40.756850 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e32c6c1-1e2f-43b7-9c82-8e9b9748ecba None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:20:40.758377 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:20:40.758497 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 9/17] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 24 06:20:40 2022] GET /identity/v3/users/029600e45e6b473db6751098bdba45e2 => generated 247 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:20:40.764113 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e32c6c1-1e2f-43b7-9c82-8e9b9748ecba None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:20:40.779914 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:20:40.854852 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6211fad0-1f0b-4e33-8df4-c72c543018a6 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:20:40.872088 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e32c6c1-1e2f-43b7-9c82-8e9b9748ecba None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:20:40.892870 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997fb8ce-78ae-4275-b06b-ea811c980837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:20:40.893768 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997fb8ce-78ae-4275-b06b-ea811c980837 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:40.894057 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997fb8ce-78ae-4275-b06b-ea811c980837 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:40.894331 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997fb8ce-78ae-4275-b06b-ea811c980837 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:40.906098 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8f1566-bd90-4b8a-9d86-54db0dc68ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:20:40.906725 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8f1566-bd90-4b8a-9d86-54db0dc68ee3 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:40.907003 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8f1566-bd90-4b8a-9d86-54db0dc68ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:40.907270 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8f1566-bd90-4b8a-9d86-54db0dc68ee3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:40.909102 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8f1566-bd90-4b8a-9d86-54db0dc68ee3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:20:40.910581 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8f1566-bd90-4b8a-9d86-54db0dc68ee3 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:20:40.912115 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 10/18] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:20:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:20:40.914165 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997fb8ce-78ae-4275-b06b-ea811c980837 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:20:40.915637 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997fb8ce-78ae-4275-b06b-ea811c980837 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:20:40.918002 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8f1566-bd90-4b8a-9d86-54db0dc68ee3 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:20:40.934547 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-997fb8ce-78ae-4275-b06b-ea811c980837 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:20:40.934547 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:20:40.934547 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:20:40.934547 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:20:40.934547 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:20:40.934547 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:20:40.934547 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:20:40.934547 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:20:40.934547 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:20:40.934547 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:20:40.934547 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:20:40.934547 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:20:40.934547 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:20:40.934547 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:20:40.934547 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:20:40.934547 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:20:40.935643 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:20:40.935643 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:20:40.935643 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:20:40.935643 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:20:40.935643 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:20:40.935643 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:20:40.935643 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:20:40.935643 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:20:40.935643 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:20:40.935643 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:20:40.935643 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:20:40.935643 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:20:40.935643 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:20:40.935643 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:20:40.935643 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:20:40.935643 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:20:40.935643 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:20:40.936652 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:20:40.936652 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:20:40.936652 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:20:40.936652 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 9/19] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:20:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:20:40.941439 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997fb8ce-78ae-4275-b06b-ea811c980837 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:20:40.998092 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:20:41.002314 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:20:41.022917 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8f1566-bd90-4b8a-9d86-54db0dc68ee3 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:20:41.043724 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997fb8ce-78ae-4275-b06b-ea811c980837 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:20:41.059196 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b555a703-e2bb-48f0-86ce-47ee2b2a99ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:20:41.059814 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b555a703-e2bb-48f0-86ce-47ee2b2a99ee None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:41.060101 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b555a703-e2bb-48f0-86ce-47ee2b2a99ee None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:41.060380 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b555a703-e2bb-48f0-86ce-47ee2b2a99ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:41.061110 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b555a703-e2bb-48f0-86ce-47ee2b2a99ee None admin] RBAC: Adding query filter params (role.id=486144ad1bcd417e86acdbc1d14327bf, scope.domain.id=default, user.id=029600e45e6b473db6751098bdba45e2) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:20:41.061454 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b555a703-e2bb-48f0-86ce-47ee2b2a99ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:20:41.062908 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b555a703-e2bb-48f0-86ce-47ee2b2a99ee None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:20:41.069718 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 11/20] 10.222.0.47 () {56 vars in 1322 bytes} [Wed Aug 24 06:20:40 2022] GET /identity/v3/role_assignments?user.id=029600e45e6b473db6751098bdba45e2&scope.domain.id=default&role.id=486144ad1bcd417e86acdbc1d14327bf => generated 233 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:20:41.080595 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96b4e7e-e4a7-4d8f-8f9c-86e1534c3eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:20:41.081268 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96b4e7e-e4a7-4d8f-8f9c-86e1534c3eaf None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:41.081551 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96b4e7e-e4a7-4d8f-8f9c-86e1534c3eaf None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:41.081825 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96b4e7e-e4a7-4d8f-8f9c-86e1534c3eaf None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:41.082497 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96b4e7e-e4a7-4d8f-8f9c-86e1534c3eaf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:20:41.082803 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96b4e7e-e4a7-4d8f-8f9c-86e1534c3eaf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:20:41.084248 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96b4e7e-e4a7-4d8f-8f9c-86e1534c3eaf None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:20:41.106338 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 10/21] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 24 06:20:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:20:43.222942 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a7c6f-d85b-41ea-a017-1dbc71cb3fb2 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:43.224001 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a7c6f-d85b-41ea-a017-1dbc71cb3fb2 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:43.224497 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a7c6f-d85b-41ea-a017-1dbc71cb3fb2 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:43.226086 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 12/22] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:20:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:20:43.231956 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1d1f29-e7f2-4961-9622-50218638c224 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:43.232831 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1d1f29-e7f2-4961-9622-50218638c224 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:43.233335 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1d1f29-e7f2-4961-9622-50218638c224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:43.262567 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-ae1d1f29-e7f2-4961-9622-50218638c224 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:20:43.264145 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-ae1d1f29-e7f2-4961-9622-50218638c224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:20:43.316094 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6565b845-e1fe-440b-8950-573d019eedf5 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:43.316892 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6565b845-e1fe-440b-8950-573d019eedf5 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:43.317595 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6565b845-e1fe-440b-8950-573d019eedf5 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:43.319777 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 13/23] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:20:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 06:20:43.329207 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d404c39-0d2c-47f0-9959-d25063dad7bb None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:43.330532 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d404c39-0d2c-47f0-9959-d25063dad7bb None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:43.331063 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d404c39-0d2c-47f0-9959-d25063dad7bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:43.356942 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-2d404c39-0d2c-47f0-9959-d25063dad7bb None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:20:43.359305 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-2d404c39-0d2c-47f0-9959-d25063dad7bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:20:43.397830 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 11/24] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:20:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 06:20:43.491611 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 14/25] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:20:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 06:20:43.542250 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8007bca3-fbea-42b7-a9d0-881a0ee8297c None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:43.542810 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8007bca3-fbea-42b7-a9d0-881a0ee8297c None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:43.543283 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8007bca3-fbea-42b7-a9d0-881a0ee8297c None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:43.544905 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 12/26] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:20:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 06:20:43.559744 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a426cda1-4d71-4692-9f24-0ba2cd89422c None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:43.560120 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a426cda1-4d71-4692-9f24-0ba2cd89422c None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:43.560491 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a426cda1-4d71-4692-9f24-0ba2cd89422c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:43.586271 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-a426cda1-4d71-4692-9f24-0ba2cd89422c None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:20:43.587506 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-a426cda1-4d71-4692-9f24-0ba2cd89422c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:20:43.683472 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 15/27] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:20:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:20:43.876720 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8007bca3-fbea-42b7-a9d0-881a0ee8297c None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:20:43.891173 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a426cda1-4d71-4692-9f24-0ba2cd89422c None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:20:43.898344 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:20:43.949702 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:20:44.057888 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8007bca3-fbea-42b7-a9d0-881a0ee8297c None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:20:44.128261 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a426cda1-4d71-4692-9f24-0ba2cd89422c None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:20:44.130863 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f61a0b-dca1-4f46-8243-9ed76a920f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:20:44.131514 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f61a0b-dca1-4f46-8243-9ed76a920f04 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:44.131790 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f61a0b-dca1-4f46-8243-9ed76a920f04 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:44.132054 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f61a0b-dca1-4f46-8243-9ed76a920f04 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:44.153714 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f61a0b-dca1-4f46-8243-9ed76a920f04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:20:44.161612 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-d7f61a0b-dca1-4f46-8243-9ed76a920f04 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:20:44.161612 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:20:44.161612 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:20:44.161612 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:20:44.161612 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:20:44.161612 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:20:44.161612 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:20:44.161612 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:20:44.161612 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:20:44.161612 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:20:44.161612 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:20:44.161612 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:20:44.161612 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:20:44.161612 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:20:44.161612 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:20:44.161612 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:20:44.162502 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:20:44.162502 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:20:44.162502 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:20:44.162502 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:20:44.162502 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:20:44.162502 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:20:44.162502 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:20:44.162502 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:20:44.162502 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:20:44.162502 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:20:44.162502 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:20:44.162502 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:20:44.162502 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:20:44.162502 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:20:44.162502 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:20:44.162502 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:20:44.163247 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:20:44.163247 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:20:44.163247 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:20:44.163247 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:20:44.163247 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:20:44.163247 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:20:44.163247 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:20:44.163247 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:20:44.163247 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:20:44.163247 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:20:44.163247 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:20:44.163247 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:20:44.165669 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 13/28] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Aug 24 06:20:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:20:44.166768 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f61a0b-dca1-4f46-8243-9ed76a920f04 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:20:44.201626 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362a667f-aa95-4dca-b1c8-393cf9cf343f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:20:44.202674 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362a667f-aa95-4dca-b1c8-393cf9cf343f None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:44.203156 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362a667f-aa95-4dca-b1c8-393cf9cf343f None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:44.203616 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362a667f-aa95-4dca-b1c8-393cf9cf343f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:44.225622 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362a667f-aa95-4dca-b1c8-393cf9cf343f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:20:44.230789 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362a667f-aa95-4dca-b1c8-393cf9cf343f None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:20:44.259877 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-362a667f-aa95-4dca-b1c8-393cf9cf343f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:20:44.259877 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:20:44.259877 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:20:44.259877 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:20:44.259877 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:20:44.259877 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:20:44.259877 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:20:44.259877 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:20:44.259877 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:20:44.259877 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:20:44.259877 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:20:44.259877 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:20:44.259877 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:20:44.259877 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:20:44.259877 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:20:44.259877 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:20:44.260776 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:20:44.260776 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:20:44.260776 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:20:44.260776 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:20:44.260776 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:20:44.260776 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:20:44.260776 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:20:44.260776 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:20:44.260776 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:20:44.260776 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:20:44.260776 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:20:44.260776 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:20:44.260776 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:20:44.260776 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:20:44.260776 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:20:44.260776 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:20:44.260776 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:20:44.261642 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:20:44.261642 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:20:44.261642 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:20:44.269618 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 16/29] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:20:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 377 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:20:44.271488 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362a667f-aa95-4dca-b1c8-393cf9cf343f None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:20:44.350713 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:20:44.363270 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f61a0b-dca1-4f46-8243-9ed76a920f04 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:20:44.389858 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:20:44.401510 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362a667f-aa95-4dca-b1c8-393cf9cf343f None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:20:44.421046 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c784cd0c-fb89-4460-81ee-a5b256cc8040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:20:44.421702 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c784cd0c-fb89-4460-81ee-a5b256cc8040 None admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:44.421987 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c784cd0c-fb89-4460-81ee-a5b256cc8040 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:44.422372 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c784cd0c-fb89-4460-81ee-a5b256cc8040 None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:44.423089 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c784cd0c-fb89-4460-81ee-a5b256cc8040 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:20:44.424575 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c784cd0c-fb89-4460-81ee-a5b256cc8040 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:20:44.441688 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ece6cd5-77b9-4c37-93ac-79fbb599abdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:20:44.442299 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ece6cd5-77b9-4c37-93ac-79fbb599abdf None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:44.442578 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ece6cd5-77b9-4c37-93ac-79fbb599abdf None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:44.442847 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ece6cd5-77b9-4c37-93ac-79fbb599abdf None admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:44.443511 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ece6cd5-77b9-4c37-93ac-79fbb599abdf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:20:44.443824 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ece6cd5-77b9-4c37-93ac-79fbb599abdf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:20:44.445219 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ece6cd5-77b9-4c37-93ac-79fbb599abdf None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:20:44.456228 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 17/30] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 24 06:20:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:20:44.456228 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ece6cd5-77b9-4c37-93ac-79fbb599abdf None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:20:44.578775 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ece6cd5-77b9-4c37-93ac-79fbb599abdf None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:20:44.600627 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:20:44.618732 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dfe163-651e-4c43-8f3f-913c1591f119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:20:44.619356 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dfe163-651e-4c43-8f3f-913c1591f119 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:44.619635 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dfe163-651e-4c43-8f3f-913c1591f119 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:44.619906 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dfe163-651e-4c43-8f3f-913c1591f119 None admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:44.620624 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dfe163-651e-4c43-8f3f-913c1591f119 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:20:44.621230 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dfe163-651e-4c43-8f3f-913c1591f119 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:20:44.622612 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dfe163-651e-4c43-8f3f-913c1591f119 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:20:44.647678 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 18/31] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 24 06:20:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:20:44.647678 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ed9af6-cac8-4da4-8dec-d4cf47b97666 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:44.647678 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ed9af6-cac8-4da4-8dec-d4cf47b97666 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:44.647678 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ed9af6-cac8-4da4-8dec-d4cf47b97666 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:44.647678 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 19/32] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:20:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:20:44.648360 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ed9af6-cac8-4da4-8dec-d4cf47b97666 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:20:44.693209 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 14/33] 10.222.0.47 () {60 vars in 1131 bytes} [Wed Aug 24 06:20:44 2022] POST /identity/v3/roles => generated 219 bytes in 521 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:20:44.693209 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec835c86-5ca7-4000-a858-e575d494d4a9 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:44.693209 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec835c86-5ca7-4000-a858-e575d494d4a9 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:44.693209 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec835c86-5ca7-4000-a858-e575d494d4a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:44.714931 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-ec835c86-5ca7-4000-a858-e575d494d4a9 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:20:44.720850 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-ec835c86-5ca7-4000-a858-e575d494d4a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:20:44.853965 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 15/34] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:20:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:20:44.854900 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ed9af6-cac8-4da4-8dec-d4cf47b97666 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:20:44.855068 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec835c86-5ca7-4000-a858-e575d494d4a9 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:20:44.930037 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570eaac0-3a4e-4d74-80a0-5bf3153579d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:20:44.930631 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570eaac0-3a4e-4d74-80a0-5bf3153579d2 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:44.930920 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570eaac0-3a4e-4d74-80a0-5bf3153579d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:44.931201 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570eaac0-3a4e-4d74-80a0-5bf3153579d2 None admin] PATH_INFO: `/v3/users/029600e45e6b473db6751098bdba45e2` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:44.932407 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570eaac0-3a4e-4d74-80a0-5bf3153579d2 None admin] RBAC: Authorizing `identity:get_user(user_id=029600e45e6b473db6751098bdba45e2)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:20:44.933895 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570eaac0-3a4e-4d74-80a0-5bf3153579d2 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:20:44.935807 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 20/35] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 24 06:20:44 2022] GET /identity/v3/users/029600e45e6b473db6751098bdba45e2 => generated 247 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:20:44.940890 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570eaac0-3a4e-4d74-80a0-5bf3153579d2 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:20:44.958497 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:20:44.964459 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec835c86-5ca7-4000-a858-e575d494d4a9 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:20:45.006294 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905fe0e2-5a55-44ac-a4f4-dfeb0d76bc1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:20:45.007103 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905fe0e2-5a55-44ac-a4f4-dfeb0d76bc1f None admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:45.007393 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905fe0e2-5a55-44ac-a4f4-dfeb0d76bc1f None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:45.007670 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905fe0e2-5a55-44ac-a4f4-dfeb0d76bc1f None admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:45.008780 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905fe0e2-5a55-44ac-a4f4-dfeb0d76bc1f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:20:45.009314 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:20:45.011426 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905fe0e2-5a55-44ac-a4f4-dfeb0d76bc1f None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:20:45.085289 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570eaac0-3a4e-4d74-80a0-5bf3153579d2 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:20:45.123226 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbe9c41-6690-4d89-a6fc-2b166027f3b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:20:45.124151 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbe9c41-6690-4d89-a6fc-2b166027f3b3 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:45.124641 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbe9c41-6690-4d89-a6fc-2b166027f3b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:45.125106 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbe9c41-6690-4d89-a6fc-2b166027f3b3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:45.126956 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbe9c41-6690-4d89-a6fc-2b166027f3b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:20:45.128739 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbe9c41-6690-4d89-a6fc-2b166027f3b3 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:20:45.130622 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 21/36] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:20:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:20:45.131973 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2040f52-4b40-45bd-8af5-e103be3e6e1f None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:45.132425 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2040f52-4b40-45bd-8af5-e103be3e6e1f None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:45.132862 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2040f52-4b40-45bd-8af5-e103be3e6e1f None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:45.134551 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 22/37] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:20:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:20:45.139289 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2040f52-4b40-45bd-8af5-e103be3e6e1f None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:20:45.174826 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 16/38] 10.222.0.47 () {60 vars in 1141 bytes} [Wed Aug 24 06:20:44 2022] POST /identity/v3/projects => generated 301 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:20:45.176821 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f64ab04-f626-4734-85f2-1841bd8a69e0 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:45.177229 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f64ab04-f626-4734-85f2-1841bd8a69e0 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:45.178413 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f64ab04-f626-4734-85f2-1841bd8a69e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:45.206856 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-2f64ab04-f626-4734-85f2-1841bd8a69e0 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:20:45.208496 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-2f64ab04-f626-4734-85f2-1841bd8a69e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:20:45.266431 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2040f52-4b40-45bd-8af5-e103be3e6e1f None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:20:45.284037 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:20:45.291166 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 17/39] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:20:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:20:45.296427 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f64ab04-f626-4734-85f2-1841bd8a69e0 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:20:45.324616 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc26016-b518-4c1e-a490-455fdad6e81e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:20:45.326388 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc26016-b518-4c1e-a490-455fdad6e81e None admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:45.326813 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc26016-b518-4c1e-a490-455fdad6e81e None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:45.327225 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc26016-b518-4c1e-a490-455fdad6e81e None admin] PATH_INFO: `/v3/domains/default/users/029600e45e6b473db6751098bdba45e2/roles/486144ad1bcd417e86acdbc1d14327bf` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:45.331778 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc26016-b518-4c1e-a490-455fdad6e81e None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=029600e45e6b473db6751098bdba45e2, role_id=486144ad1bcd417e86acdbc1d14327bf)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:20:45.333768 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc26016-b518-4c1e-a490-455fdad6e81e None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:20:45.381076 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:20:45.405760 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f64ab04-f626-4734-85f2-1841bd8a69e0 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:20:45.442083 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7c0f4e-848d-4edd-9525-cf03e0763af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:20:45.442708 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7c0f4e-848d-4edd-9525-cf03e0763af1 None admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:45.442986 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7c0f4e-848d-4edd-9525-cf03e0763af1 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:45.443265 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7c0f4e-848d-4edd-9525-cf03e0763af1 None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:45.443921 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7c0f4e-848d-4edd-9525-cf03e0763af1 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:20:45.445392 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7c0f4e-848d-4edd-9525-cf03e0763af1 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:20:45.537347 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 18/40] 10.222.0.47 () {60 vars in 1131 bytes} [Wed Aug 24 06:20:45 2022] POST /identity/v3/roles => generated 225 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:20:45.543609 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f61278-16fb-4f74-9d02-0e1b975ef8a1 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:45.544580 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f61278-16fb-4f74-9d02-0e1b975ef8a1 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:45.545398 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f61278-16fb-4f74-9d02-0e1b975ef8a1 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:45.548748 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 19/41] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:20:45 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:20:45.563058 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374afc95-d6b4-43ff-bb5e-67b2b54c4d62 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:45.564366 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374afc95-d6b4-43ff-bb5e-67b2b54c4d62 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:45.564366 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374afc95-d6b4-43ff-bb5e-67b2b54c4d62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:45.586044 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-374afc95-d6b4-43ff-bb5e-67b2b54c4d62 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:20:45.587315 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-374afc95-d6b4-43ff-bb5e-67b2b54c4d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:20:45.643002 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 23/42] 10.222.0.47 () {58 vars in 1361 bytes} [Wed Aug 24 06:20:45 2022] PUT /identity/v3/domains/default/users/029600e45e6b473db6751098bdba45e2/roles/486144ad1bcd417e86acdbc1d14327bf => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:20:45.654403 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc26016-b518-4c1e-a490-455fdad6e81e None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:20:45.693380 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 20/43] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:20:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:20:45.790593 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc26016-b518-4c1e-a490-455fdad6e81e None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:20:45.833783 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb7147b-6979-4638-af73-66265530e324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:20:45.834424 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb7147b-6979-4638-af73-66265530e324 None admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:45.834708 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb7147b-6979-4638-af73-66265530e324 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:45.835079 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb7147b-6979-4638-af73-66265530e324 None admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:45.835860 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb7147b-6979-4638-af73-66265530e324 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:20:45.837520 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb7147b-6979-4638-af73-66265530e324 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:20:45.910109 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 24/44] 10.222.0.47 () {60 vars in 1131 bytes} [Wed Aug 24 06:20:45 2022] POST /identity/v3/roles => generated 223 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:20:45.976944 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374afc95-d6b4-43ff-bb5e-67b2b54c4d62 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:20:46.053518 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:20:46.091082 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374afc95-d6b4-43ff-bb5e-67b2b54c4d62 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:20:46.127497 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb476534-7659-42ad-9000-946a1b7780bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:20:46.128153 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb476534-7659-42ad-9000-946a1b7780bb None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:46.128439 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb476534-7659-42ad-9000-946a1b7780bb None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:46.128721 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb476534-7659-42ad-9000-946a1b7780bb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:46.129869 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb476534-7659-42ad-9000-946a1b7780bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:20:46.131358 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb476534-7659-42ad-9000-946a1b7780bb None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:20:46.132972 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 21/45] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:20:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:20:46.138666 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb7147b-6979-4638-af73-66265530e324 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:20:46.175147 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:20:46.241341 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb7147b-6979-4638-af73-66265530e324 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:20:46.274409 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60d6a55-b8f5-45e7-9632-3c9f86ee344d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:20:46.275035 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60d6a55-b8f5-45e7-9632-3c9f86ee344d None admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:46.275291 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60d6a55-b8f5-45e7-9632-3c9f86ee344d None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:46.275565 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60d6a55-b8f5-45e7-9632-3c9f86ee344d None admin] PATH_INFO: `/v3/projects` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:46.276202 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60d6a55-b8f5-45e7-9632-3c9f86ee344d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:20:46.277690 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60d6a55-b8f5-45e7-9632-3c9f86ee344d None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:20:46.368121 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 25/46] 10.222.0.47 () {60 vars in 1141 bytes} [Wed Aug 24 06:20:46 2022] POST /identity/v3/projects => generated 312 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:20:47.631803 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275f1694-99d6-4929-bd21-d90464e44f9c None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:47.632212 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275f1694-99d6-4929-bd21-d90464e44f9c None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:47.632363 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275f1694-99d6-4929-bd21-d90464e44f9c None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:47.633753 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 22/47] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:20:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:20:47.639003 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657dc582-8f3b-4628-8cb8-ec46c928a11e None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:47.639383 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657dc582-8f3b-4628-8cb8-ec46c928a11e None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:47.639567 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657dc582-8f3b-4628-8cb8-ec46c928a11e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:47.659232 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-657dc582-8f3b-4628-8cb8-ec46c928a11e None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:20:47.660777 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-657dc582-8f3b-4628-8cb8-ec46c928a11e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:20:47.736725 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 26/48] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:20:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:20:48.009118 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275f1694-99d6-4929-bd21-d90464e44f9c None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:20:48.111339 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275f1694-99d6-4929-bd21-d90464e44f9c None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:20:48.145128 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f71c98e-36a0-4888-abd4-a73740e0f958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:20:48.145792 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f71c98e-36a0-4888-abd4-a73740e0f958 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:48.146078 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f71c98e-36a0-4888-abd4-a73740e0f958 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:48.146354 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f71c98e-36a0-4888-abd4-a73740e0f958 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:48.149098 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:20:48.158162 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f71c98e-36a0-4888-abd4-a73740e0f958 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:20:48.163932 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-8f71c98e-36a0-4888-abd4-a73740e0f958 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:20:48.163932 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:20:48.163932 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:20:48.163932 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:20:48.163932 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:20:48.163932 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:20:48.163932 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:20:48.163932 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:20:48.163932 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:20:48.163932 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:20:48.163932 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:20:48.163932 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:20:48.163932 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:20:48.163932 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:20:48.163932 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:20:48.163932 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:20:48.164747 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:20:48.164747 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:20:48.164747 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:20:48.164747 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:20:48.164747 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:20:48.164747 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:20:48.164747 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:20:48.164747 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:20:48.164747 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:20:48.164747 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:20:48.164747 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:20:48.164747 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:20:48.164747 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:20:48.164747 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:20:48.164747 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:20:48.164747 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:20:48.165524 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:20:48.165524 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:20:48.165524 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:20:48.165524 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:20:48.165524 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:20:48.165524 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:20:48.165524 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:20:48.165524 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:20:48.165524 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:20:48.165524 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:20:48.165524 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:20:48.165524 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:20:48.165524 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 23/49] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Aug 24 06:20:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:20:48.169962 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657dc582-8f3b-4628-8cb8-ec46c928a11e None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:20:48.188118 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:20:48.272971 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657dc582-8f3b-4628-8cb8-ec46c928a11e None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:20:48.303357 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5269c53e-7f41-4040-8e0e-049211cf427e None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:48.303650 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5269c53e-7f41-4040-8e0e-049211cf427e None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:48.303933 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5269c53e-7f41-4040-8e0e-049211cf427e None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:48.305283 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 24/50] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:20:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 06:20:48.308599 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f885d2a0-af63-445d-9908-1d46434fb9ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:20:48.309243 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f885d2a0-af63-445d-9908-1d46434fb9ca None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:48.309532 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f885d2a0-af63-445d-9908-1d46434fb9ca None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:48.309810 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f885d2a0-af63-445d-9908-1d46434fb9ca None admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:48.310271 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d5ba4d-7b13-42ea-9baa-82682cc2461d None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:48.310483 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f885d2a0-af63-445d-9908-1d46434fb9ca None admin] RBAC: Adding query filter params (name=admin) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:20:48.310580 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d5ba4d-7b13-42ea-9baa-82682cc2461d None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:48.310789 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f885d2a0-af63-445d-9908-1d46434fb9ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:20:48.310885 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d5ba4d-7b13-42ea-9baa-82682cc2461d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:48.312179 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f885d2a0-af63-445d-9908-1d46434fb9ca None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:20:48.321109 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 27/51] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 24 06:20:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:20:48.326503 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f885d2a0-af63-445d-9908-1d46434fb9ca None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:20:48.328525 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-b9d5ba4d-7b13-42ea-9baa-82682cc2461d None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:20:48.329862 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-b9d5ba4d-7b13-42ea-9baa-82682cc2461d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:20:48.397932 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 25/52] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:20:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:20:48.407538 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:20:48.429331 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f885d2a0-af63-445d-9908-1d46434fb9ca None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:20:48.465164 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3253f104-e566-4812-850e-19136b0d158e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:20:48.465824 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3253f104-e566-4812-850e-19136b0d158e None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:48.466116 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3253f104-e566-4812-850e-19136b0d158e None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:48.466394 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3253f104-e566-4812-850e-19136b0d158e None admin] PATH_INFO: `/v3/users/029600e45e6b473db6751098bdba45e2` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:48.467556 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3253f104-e566-4812-850e-19136b0d158e None admin] RBAC: Authorizing `identity:get_user(user_id=029600e45e6b473db6751098bdba45e2)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:20:48.469019 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3253f104-e566-4812-850e-19136b0d158e None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:20:48.471743 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 28/53] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 24 06:20:48 2022] GET /identity/v3/users/029600e45e6b473db6751098bdba45e2 => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:20:48.477247 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d5ba4d-7b13-42ea-9baa-82682cc2461d None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:20:48.577566 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d5ba4d-7b13-42ea-9baa-82682cc2461d None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:20:48.604720 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:20:48.613691 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497c32c2-faf4-42ac-8765-2d8108a3fdaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:20:48.614364 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c32c2-faf4-42ac-8765-2d8108a3fdaf None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:48.614646 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c32c2-faf4-42ac-8765-2d8108a3fdaf None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:48.614919 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c32c2-faf4-42ac-8765-2d8108a3fdaf None admin] PATH_INFO: `/v3/domains/default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:48.616022 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497c32c2-faf4-42ac-8765-2d8108a3fdaf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:20:48.617484 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497c32c2-faf4-42ac-8765-2d8108a3fdaf None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:20:48.619147 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 26/54] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:20:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:20:48.629157 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3253f104-e566-4812-850e-19136b0d158e None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:20:48.673984 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497c32c2-faf4-42ac-8765-2d8108a3fdaf None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:20:48.736523 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3253f104-e566-4812-850e-19136b0d158e None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:20:48.772358 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1aaa51-5dc7-4709-93b8-236a18beb896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:20:48.772965 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1aaa51-5dc7-4709-93b8-236a18beb896 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:48.773282 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1aaa51-5dc7-4709-93b8-236a18beb896 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:48.773562 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1aaa51-5dc7-4709-93b8-236a18beb896 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:48.774299 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1aaa51-5dc7-4709-93b8-236a18beb896 None admin] RBAC: Adding query filter params (role.id=486144ad1bcd417e86acdbc1d14327bf, scope.domain.id=default, user.id=029600e45e6b473db6751098bdba45e2) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:20:48.774609 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1aaa51-5dc7-4709-93b8-236a18beb896 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:20:48.775033 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497c32c2-faf4-42ac-8765-2d8108a3fdaf None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:20:48.776051 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1aaa51-5dc7-4709-93b8-236a18beb896 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:20:48.782321 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 29/55] 10.222.0.47 () {56 vars in 1322 bytes} [Wed Aug 24 06:20:48 2022] GET /identity/v3/role_assignments?user.id=029600e45e6b473db6751098bdba45e2&scope.domain.id=default&role.id=486144ad1bcd417e86acdbc1d14327bf => generated 530 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:20:48.814786 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d52c72-3032-4711-a128-b09bcd3fe4a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:20:48.815488 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d52c72-3032-4711-a128-b09bcd3fe4a9 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:48.815777 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d52c72-3032-4711-a128-b09bcd3fe4a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:48.816057 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d52c72-3032-4711-a128-b09bcd3fe4a9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:48.817412 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d52c72-3032-4711-a128-b09bcd3fe4a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:20:48.818923 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d52c72-3032-4711-a128-b09bcd3fe4a9 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:20:48.820573 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 27/56] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:20:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:20:48.826530 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1aaa51-5dc7-4709-93b8-236a18beb896 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:20:48.842034 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:20:48.941069 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1aaa51-5dc7-4709-93b8-236a18beb896 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:20:48.976795 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857a5593-c2e5-4c06-a012-b6f780f058da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:20:48.977446 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857a5593-c2e5-4c06-a012-b6f780f058da None admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:48.977745 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857a5593-c2e5-4c06-a012-b6f780f058da None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:48.978026 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857a5593-c2e5-4c06-a012-b6f780f058da None admin] PATH_INFO: `/v3/projects` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:48.978685 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857a5593-c2e5-4c06-a012-b6f780f058da None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:20:48.980165 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857a5593-c2e5-4c06-a012-b6f780f058da None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:20:49.080200 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 30/57] 10.222.0.47 () {60 vars in 1140 bytes} [Wed Aug 24 06:20:48 2022] POST /identity/v3/projects => generated 298 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:20:51.235537 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5d8a64-e207-4e5a-ac1d-330620d3915b None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:51.236082 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5d8a64-e207-4e5a-ac1d-330620d3915b None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:51.236082 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5d8a64-e207-4e5a-ac1d-330620d3915b None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:51.237474 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 28/58] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:20:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:20:51.242852 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f3d5e2-68b9-4cd2-86d7-88d74889b3a7 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:51.243266 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f3d5e2-68b9-4cd2-86d7-88d74889b3a7 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:51.243412 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f3d5e2-68b9-4cd2-86d7-88d74889b3a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:51.261662 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-77f3d5e2-68b9-4cd2-86d7-88d74889b3a7 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:20:51.262861 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-77f3d5e2-68b9-4cd2-86d7-88d74889b3a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:20:51.319054 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:20:51.330099 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 31/59] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:20:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:20:51.603935 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5d8a64-e207-4e5a-ac1d-330620d3915b None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:20:51.666381 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:20:51.705788 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5d8a64-e207-4e5a-ac1d-330620d3915b None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:20:51.739954 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e0ea55-1290-4c75-bd3d-af341c80ab5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:20:51.740587 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e0ea55-1290-4c75-bd3d-af341c80ab5f None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:51.740869 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e0ea55-1290-4c75-bd3d-af341c80ab5f None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:51.741139 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e0ea55-1290-4c75-bd3d-af341c80ab5f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:51.742277 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e0ea55-1290-4c75-bd3d-af341c80ab5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:20:51.743710 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e0ea55-1290-4c75-bd3d-af341c80ab5f None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:20:51.745299 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 29/60] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:20:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:20:51.751481 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f3d5e2-68b9-4cd2-86d7-88d74889b3a7 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:20:51.854132 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f3d5e2-68b9-4cd2-86d7-88d74889b3a7 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:20:51.890989 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6315fe-5ea5-4953-92a2-c34eda89d31d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:20:51.891620 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6315fe-5ea5-4953-92a2-c34eda89d31d None admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:51.891973 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6315fe-5ea5-4953-92a2-c34eda89d31d None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:51.892277 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6315fe-5ea5-4953-92a2-c34eda89d31d None admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:51.892980 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6315fe-5ea5-4953-92a2-c34eda89d31d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:20:51.894500 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6315fe-5ea5-4953-92a2-c34eda89d31d None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:20:52.001813 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 32/61] 10.222.0.47 () {60 vars in 1132 bytes} [Wed Aug 24 06:20:51 2022] POST /identity/v3/users => generated 275 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:20:55.209195 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d787e2-8bbf-4e3e-a019-b4d866d6e70a None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:55.210219 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d787e2-8bbf-4e3e-a019-b4d866d6e70a None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:55.210702 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d787e2-8bbf-4e3e-a019-b4d866d6e70a None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:55.212236 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 30/62] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:20:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:20:55.217633 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9374e41-c933-4273-b939-03d8b8c9ffab None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:55.218431 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9374e41-c933-4273-b939-03d8b8c9ffab None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:55.218912 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9374e41-c933-4273-b939-03d8b8c9ffab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:55.242720 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5675cc9-10c5-40ea-8e39-985be6831ba5 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:55.243317 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5675cc9-10c5-40ea-8e39-985be6831ba5 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:55.243806 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5675cc9-10c5-40ea-8e39-985be6831ba5 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:55.245748 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 31/63] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:20:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:20:55.251695 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4896d3-27ac-4ea9-8569-7a0fc0586721 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:55.252272 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4896d3-27ac-4ea9-8569-7a0fc0586721 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:55.252743 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4896d3-27ac-4ea9-8569-7a0fc0586721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:55.257585 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-a9374e41-c933-4273-b939-03d8b8c9ffab None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:20:55.260180 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-a9374e41-c933-4273-b939-03d8b8c9ffab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:20:55.287936 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-8a4896d3-27ac-4ea9-8569-7a0fc0586721 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:20:55.289549 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-8a4896d3-27ac-4ea9-8569-7a0fc0586721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:20:55.348606 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:20:55.385206 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 32/64] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:20:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 06:20:55.385206 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c424120-6de8-415a-bf4b-50b30dca052e None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:55.385206 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c424120-6de8-415a-bf4b-50b30dca052e None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:55.385206 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c424120-6de8-415a-bf4b-50b30dca052e None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:55.385206 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 33/65] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:20:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:20:55.385925 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 33/66] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:20:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:20:55.389543 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852cc966-6c96-468b-bdea-832298b5b1cb None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:55.389832 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852cc966-6c96-468b-bdea-832298b5b1cb None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:55.390112 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852cc966-6c96-468b-bdea-832298b5b1cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:55.410890 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-852cc966-6c96-468b-bdea-832298b5b1cb None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:20:55.412105 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-852cc966-6c96-468b-bdea-832298b5b1cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:20:55.472968 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:20:55.487958 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 34/67] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:20:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 06:20:55.730367 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546784ab-94e3-41a7-b849-95dc03be7a4e None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:55.730580 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546784ab-94e3-41a7-b849-95dc03be7a4e None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:55.730842 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546784ab-94e3-41a7-b849-95dc03be7a4e None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:55.732511 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 34/68] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:20:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:20:55.737764 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd77221a-d66b-4f45-ba1f-06fab2b54426 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:55.738050 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd77221a-d66b-4f45-ba1f-06fab2b54426 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:55.738325 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd77221a-d66b-4f45-ba1f-06fab2b54426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:55.770269 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-dd77221a-d66b-4f45-ba1f-06fab2b54426 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:20:55.771877 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-dd77221a-d66b-4f45-ba1f-06fab2b54426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:20:55.842045 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546784ab-94e3-41a7-b849-95dc03be7a4e None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:20:55.848324 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 35/69] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:20:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:20:55.856647 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd77221a-d66b-4f45-ba1f-06fab2b54426 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:20:55.962553 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546784ab-94e3-41a7-b849-95dc03be7a4e None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:20:55.988935 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd77221a-d66b-4f45-ba1f-06fab2b54426 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:20:56.009229 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd94697-038e-4d9d-97c1-e15b9c049bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:20:56.009884 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd94697-038e-4d9d-97c1-e15b9c049bf7 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:56.010651 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:20:56.011657 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd94697-038e-4d9d-97c1-e15b9c049bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:56.011944 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd94697-038e-4d9d-97c1-e15b9c049bf7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:56.026185 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351f3550-b85c-41de-9a14-15c5972ac38f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:20:56.026481 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd94697-038e-4d9d-97c1-e15b9c049bf7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:20:56.026896 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351f3550-b85c-41de-9a14-15c5972ac38f None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:56.027180 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351f3550-b85c-41de-9a14-15c5972ac38f None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:56.027620 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351f3550-b85c-41de-9a14-15c5972ac38f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:56.032600 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-3bd94697-038e-4d9d-97c1-e15b9c049bf7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 06:20:56.032600 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:20:56.032600 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:20:56.032600 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:20:56.032600 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:20:56.032600 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:20:56.032600 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:20:56.032600 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:20:56.032600 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:20:56.032600 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:20:56.032600 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:20:56.032600 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:20:56.032600 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:20:56.032600 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:20:56.032600 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:20:56.032600 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:20:56.034356 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:20:56.034356 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:20:56.034356 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:20:56.034356 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:20:56.034356 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:20:56.034356 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:20:56.034356 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:20:56.034356 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:20:56.034356 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:20:56.034356 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:20:56.034356 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:20:56.034356 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:20:56.034356 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:20:56.034356 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:20:56.034356 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:20:56.034356 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:20:56.035268 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:20:56.035268 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:20:56.035268 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:20:56.035268 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:20:56.035268 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:20:56.035268 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:20:56.035268 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:20:56.035268 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:20:56.035268 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:20:56.035268 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:20:56.035268 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 06:20:56.035268 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:20:56.035268 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 35/70] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 24 06:20:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:20:56.038024 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd94697-038e-4d9d-97c1-e15b9c049bf7 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:20:56.047334 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351f3550-b85c-41de-9a14-15c5972ac38f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:20:56.052630 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-351f3550-b85c-41de-9a14-15c5972ac38f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 06:20:56.052630 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:20:56.052630 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:20:56.052630 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:20:56.052630 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:20:56.052630 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:20:56.052630 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:20:56.052630 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:20:56.052630 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:20:56.052630 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:20:56.052630 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:20:56.052630 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:20:56.052630 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:20:56.052630 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:20:56.052630 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:20:56.052630 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:20:56.055000 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:20:56.055000 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:20:56.055000 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:20:56.055000 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:20:56.055000 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:20:56.055000 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:20:56.055000 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:20:56.055000 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:20:56.055000 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:20:56.055000 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:20:56.055000 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:20:56.055000 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:20:56.055000 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:20:56.055000 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:20:56.055000 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:20:56.055000 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:20:56.056920 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:20:56.056920 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:20:56.056920 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:20:56.056920 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:20:56.056920 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:20:56.056920 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:20:56.056920 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:20:56.056920 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:20:56.056920 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:20:56.056920 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:20:56.056920 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 06:20:56.056920 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:20:56.056920 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 36/71] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Aug 24 06:20:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:20:56.056920 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351f3550-b85c-41de-9a14-15c5972ac38f None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:20:56.095796 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:20:56.164466 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351f3550-b85c-41de-9a14-15c5972ac38f None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:20:56.174884 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd94697-038e-4d9d-97c1-e15b9c049bf7 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:20:56.200553 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ace2a18-ed4f-43a9-a3e0-1f8c935ee86a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:20:56.201174 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ace2a18-ed4f-43a9-a3e0-1f8c935ee86a None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:56.201484 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ace2a18-ed4f-43a9-a3e0-1f8c935ee86a None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:56.201761 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ace2a18-ed4f-43a9-a3e0-1f8c935ee86a None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:56.202433 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ace2a18-ed4f-43a9-a3e0-1f8c935ee86a None admin] RBAC: Adding query filter params (name=member) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:20:56.202881 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ace2a18-ed4f-43a9-a3e0-1f8c935ee86a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:20:56.204976 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ace2a18-ed4f-43a9-a3e0-1f8c935ee86a None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:20:56.215556 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd750b9-a7d6-44dd-8572-e1266720a945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:20:56.216192 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd750b9-a7d6-44dd-8572-e1266720a945 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:56.216474 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd750b9-a7d6-44dd-8572-e1266720a945 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:56.216746 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd750b9-a7d6-44dd-8572-e1266720a945 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:56.223276 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 37/72] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Aug 24 06:20:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:20:56.223276 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ace2a18-ed4f-43a9-a3e0-1f8c935ee86a None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:20:56.233150 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd750b9-a7d6-44dd-8572-e1266720a945 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:20:56.238219 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-bcd750b9-a7d6-44dd-8572-e1266720a945 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:20:56.238219 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:20:56.238219 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:20:56.238219 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:20:56.238219 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:20:56.238219 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:20:56.238219 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:20:56.238219 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:20:56.238219 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:20:56.238219 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:20:56.238219 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:20:56.238219 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:20:56.238219 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:20:56.238219 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:20:56.238219 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:20:56.238219 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:20:56.239085 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:20:56.239085 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:20:56.239085 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:20:56.239085 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:20:56.239085 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:20:56.239085 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:20:56.239085 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:20:56.239085 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:20:56.239085 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:20:56.239085 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:20:56.239085 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:20:56.239085 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:20:56.239085 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:20:56.239085 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:20:56.239085 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:20:56.239085 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:20:56.239871 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:20:56.239871 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:20:56.239871 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:20:56.239871 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:20:56.239871 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:20:56.239871 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:20:56.239871 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:20:56.239871 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:20:56.239871 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:20:56.239871 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:20:56.239871 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:20:56.239871 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:20:56.239871 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 36/73] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Aug 24 06:20:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:20:56.240501 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd750b9-a7d6-44dd-8572-e1266720a945 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:20:56.276158 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:20:56.337748 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ace2a18-ed4f-43a9-a3e0-1f8c935ee86a None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:20:56.340521 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:20:56.348562 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd750b9-a7d6-44dd-8572-e1266720a945 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:20:56.373054 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9b30b2-e2ce-4c3e-b896-41b9cb49ff4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:20:56.373702 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9b30b2-e2ce-4c3e-b896-41b9cb49ff4a None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:56.373991 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9b30b2-e2ce-4c3e-b896-41b9cb49ff4a None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:56.374265 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9b30b2-e2ce-4c3e-b896-41b9cb49ff4a None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:56.374939 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9b30b2-e2ce-4c3e-b896-41b9cb49ff4a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:20:56.375383 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9b30b2-e2ce-4c3e-b896-41b9cb49ff4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:20:56.377669 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9b30b2-e2ce-4c3e-b896-41b9cb49ff4a None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:20:56.386044 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 38/74] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 24 06:20:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:20:56.386722 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9b30b2-e2ce-4c3e-b896-41b9cb49ff4a None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:20:56.387346 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ae2a09-0886-4dac-96e1-7c54ea74f554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:20:56.387986 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ae2a09-0886-4dac-96e1-7c54ea74f554 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:56.388270 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ae2a09-0886-4dac-96e1-7c54ea74f554 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:56.388547 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ae2a09-0886-4dac-96e1-7c54ea74f554 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:56.399839 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ae2a09-0886-4dac-96e1-7c54ea74f554 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:20:56.404915 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-b4ae2a09-0886-4dac-96e1-7c54ea74f554 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 06:20:56.404915 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:20:56.404915 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:20:56.404915 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:20:56.404915 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:20:56.404915 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:20:56.404915 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:20:56.404915 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:20:56.404915 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:20:56.404915 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:20:56.404915 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:20:56.404915 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:20:56.404915 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:20:56.404915 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:20:56.404915 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:20:56.404915 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:20:56.405771 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:20:56.405771 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:20:56.405771 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:20:56.405771 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:20:56.405771 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:20:56.405771 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:20:56.405771 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:20:56.405771 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:20:56.405771 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:20:56.405771 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:20:56.405771 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:20:56.405771 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:20:56.405771 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:20:56.405771 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:20:56.405771 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:20:56.405771 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:20:56.411825 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:20:56.411825 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:20:56.411825 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:20:56.411825 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:20:56.411825 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:20:56.411825 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:20:56.411825 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:20:56.411825 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:20:56.411825 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:20:56.411825 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:20:56.411825 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 06:20:56.411825 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:20:56.411825 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 37/75] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 24 06:20:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:20:56.411825 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ae2a09-0886-4dac-96e1-7c54ea74f554 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:20:56.501770 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9b30b2-e2ce-4c3e-b896-41b9cb49ff4a None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:20:56.504279 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:20:56.521722 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ae2a09-0886-4dac-96e1-7c54ea74f554 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:20:56.540661 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c9ba2d-359d-495e-8025-6d31a77371fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:20:56.541358 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c9ba2d-359d-495e-8025-6d31a77371fb None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:56.541648 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c9ba2d-359d-495e-8025-6d31a77371fb None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:56.541930 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c9ba2d-359d-495e-8025-6d31a77371fb None admin] PATH_INFO: `/v3/users/b751994501284709b7f528880544d3b4` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:56.559465 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91a9b98-c5cb-42af-a384-33cb938a4f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:20:56.560085 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91a9b98-c5cb-42af-a384-33cb938a4f4a None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:56.560401 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91a9b98-c5cb-42af-a384-33cb938a4f4a None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:56.560684 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91a9b98-c5cb-42af-a384-33cb938a4f4a None admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:56.561389 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91a9b98-c5cb-42af-a384-33cb938a4f4a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:20:56.561700 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91a9b98-c5cb-42af-a384-33cb938a4f4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:20:56.563053 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c9ba2d-359d-495e-8025-6d31a77371fb None admin] RBAC: Authorizing `identity:get_user(user_id=b751994501284709b7f528880544d3b4)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:20:56.563241 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91a9b98-c5cb-42af-a384-33cb938a4f4a None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:20:56.565216 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c9ba2d-359d-495e-8025-6d31a77371fb None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:20:56.567498 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 39/76] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 24 06:20:56 2022] GET /identity/v3/users/b751994501284709b7f528880544d3b4 => generated 275 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:20:56.572896 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c9ba2d-359d-495e-8025-6d31a77371fb None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:20:56.573391 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 38/77] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 24 06:20:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:20:56.574480 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91a9b98-c5cb-42af-a384-33cb938a4f4a None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:20:56.605794 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:20:56.685329 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c9ba2d-359d-495e-8025-6d31a77371fb None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:20:56.690511 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91a9b98-c5cb-42af-a384-33cb938a4f4a None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:20:56.717870 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9230b636-5e48-4815-8892-05d4beb58032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:20:56.718583 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9230b636-5e48-4815-8892-05d4beb58032 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:56.719357 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9230b636-5e48-4815-8892-05d4beb58032 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:56.719643 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9230b636-5e48-4815-8892-05d4beb58032 None admin] PATH_INFO: `/v3/users/b751994501284709b7f528880544d3b4` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:56.720872 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9230b636-5e48-4815-8892-05d4beb58032 None admin] RBAC: Authorizing `identity:get_user(user_id=b751994501284709b7f528880544d3b4)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:20:56.722346 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9230b636-5e48-4815-8892-05d4beb58032 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:20:56.723659 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40476f9f-1e0b-4c91-adfe-edd84006f1f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:20:56.723912 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 40/78] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 24 06:20:56 2022] GET /identity/v3/users/b751994501284709b7f528880544d3b4 => generated 275 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:20:56.724275 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40476f9f-1e0b-4c91-adfe-edd84006f1f4 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:56.724562 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40476f9f-1e0b-4c91-adfe-edd84006f1f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:56.724736 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9230b636-5e48-4815-8892-05d4beb58032 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:20:56.724992 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40476f9f-1e0b-4c91-adfe-edd84006f1f4 None admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:56.725743 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40476f9f-1e0b-4c91-adfe-edd84006f1f4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:20:56.726060 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40476f9f-1e0b-4c91-adfe-edd84006f1f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:20:56.727480 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40476f9f-1e0b-4c91-adfe-edd84006f1f4 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:20:56.736595 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 39/79] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Aug 24 06:20:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:20:56.737441 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40476f9f-1e0b-4c91-adfe-edd84006f1f4 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:20:56.755984 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:20:56.819732 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:20:56.833324 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9230b636-5e48-4815-8892-05d4beb58032 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:20:56.838262 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40476f9f-1e0b-4c91-adfe-edd84006f1f4 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:20:56.868217 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405e03c9-18ec-4722-9d75-d4eda51e9b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:20:56.868857 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405e03c9-18ec-4722-9d75-d4eda51e9b03 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:56.869143 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405e03c9-18ec-4722-9d75-d4eda51e9b03 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:56.869457 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405e03c9-18ec-4722-9d75-d4eda51e9b03 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:56.874885 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fb5d14-f73e-4a59-a768-141064a0a69a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:20:56.875416 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb5d14-f73e-4a59-a768-141064a0a69a None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:56.875780 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb5d14-f73e-4a59-a768-141064a0a69a None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:56.876073 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb5d14-f73e-4a59-a768-141064a0a69a None admin] PATH_INFO: `/v3/users/029600e45e6b473db6751098bdba45e2` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:56.877356 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fb5d14-f73e-4a59-a768-141064a0a69a None admin] RBAC: Authorizing `identity:get_user(user_id=029600e45e6b473db6751098bdba45e2)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:20:56.878843 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fb5d14-f73e-4a59-a768-141064a0a69a None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:20:56.880580 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 40/80] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 24 06:20:56 2022] GET /identity/v3/users/029600e45e6b473db6751098bdba45e2 => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:20:56.881956 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce50e6a0-7b93-4a5c-a9a1-40d4396dc0ef None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:56.882245 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce50e6a0-7b93-4a5c-a9a1-40d4396dc0ef None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:56.882529 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce50e6a0-7b93-4a5c-a9a1-40d4396dc0ef None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:56.883841 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 41/81] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:20:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:20:56.886082 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce50e6a0-7b93-4a5c-a9a1-40d4396dc0ef None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:20:56.946224 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405e03c9-18ec-4722-9d75-d4eda51e9b03 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:20:56.947828 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405e03c9-18ec-4722-9d75-d4eda51e9b03 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:20:56.994466 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce50e6a0-7b93-4a5c-a9a1-40d4396dc0ef None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:20:57.034107 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab358f4e-1063-4eab-93be-7e6c8d1c510d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:20:57.034649 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab358f4e-1063-4eab-93be-7e6c8d1c510d None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:57.034936 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab358f4e-1063-4eab-93be-7e6c8d1c510d None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:57.035214 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab358f4e-1063-4eab-93be-7e6c8d1c510d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:57.055408 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:20:57.085452 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:20:57.106282 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab358f4e-1063-4eab-93be-7e6c8d1c510d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:20:57.107741 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab358f4e-1063-4eab-93be-7e6c8d1c510d None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:20:57.157640 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 41/82] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 24 06:20:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 06:20:57.158576 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405e03c9-18ec-4722-9d75-d4eda51e9b03 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:20:57.212438 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 42/83] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 24 06:20:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 06:20:57.213314 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab358f4e-1063-4eab-93be-7e6c8d1c510d None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:20:57.261524 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405e03c9-18ec-4722-9d75-d4eda51e9b03 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:20:57.262504 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:20:57.290557 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:20:57.296082 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e8a5ea-7dd4-438b-961d-b9b9d7758f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:20:57.296729 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e8a5ea-7dd4-438b-961d-b9b9d7758f76 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:57.297020 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e8a5ea-7dd4-438b-961d-b9b9d7758f76 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:57.297917 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e8a5ea-7dd4-438b-961d-b9b9d7758f76 None admin] PATH_INFO: `/v3/users/b751994501284709b7f528880544d3b4` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:57.299177 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e8a5ea-7dd4-438b-961d-b9b9d7758f76 None admin] RBAC: Authorizing `identity:get_user(user_id=b751994501284709b7f528880544d3b4)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:20:57.300632 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e8a5ea-7dd4-438b-961d-b9b9d7758f76 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:20:57.302448 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 42/84] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 24 06:20:57 2022] GET /identity/v3/users/b751994501284709b7f528880544d3b4 => generated 275 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:20:57.307212 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b326c8-f801-4cb0-8ba2-169c44cc546d None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:57.307212 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b326c8-f801-4cb0-8ba2-169c44cc546d None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:57.307212 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b326c8-f801-4cb0-8ba2-169c44cc546d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:57.314295 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab358f4e-1063-4eab-93be-7e6c8d1c510d None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:20:57.323844 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-e7b326c8-f801-4cb0-8ba2-169c44cc546d None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:20:57.325406 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-e7b326c8-f801-4cb0-8ba2-169c44cc546d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:20:57.349062 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a679467-8ee7-445b-a9c0-b4ea4c920aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:20:57.349706 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a679467-8ee7-445b-a9c0-b4ea4c920aaf None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:57.349992 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a679467-8ee7-445b-a9c0-b4ea4c920aaf None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:57.350270 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a679467-8ee7-445b-a9c0-b4ea4c920aaf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:57.393257 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 43/85] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:20:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:20:57.393257 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b326c8-f801-4cb0-8ba2-169c44cc546d None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:20:57.419577 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a679467-8ee7-445b-a9c0-b4ea4c920aaf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:20:57.421074 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a679467-8ee7-445b-a9c0-b4ea4c920aaf None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:20:57.474694 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:20:57.499384 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b326c8-f801-4cb0-8ba2-169c44cc546d None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:20:57.526239 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:20:57.533137 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 43/86] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 24 06:20:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 06:20:57.534288 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a679467-8ee7-445b-a9c0-b4ea4c920aaf None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:20:57.538764 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1850a452-4004-4df2-a8d5-158a60e42c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:20:57.539427 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1850a452-4004-4df2-a8d5-158a60e42c0f None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:57.539714 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1850a452-4004-4df2-a8d5-158a60e42c0f None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:57.539997 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1850a452-4004-4df2-a8d5-158a60e42c0f None admin] PATH_INFO: `/v3/projects/df345274c3b840b486c303c6b301052e` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:57.541270 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1850a452-4004-4df2-a8d5-158a60e42c0f None admin] RBAC: Authorizing `identity:get_project(project_id=df345274c3b840b486c303c6b301052e)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:20:57.543101 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1850a452-4004-4df2-a8d5-158a60e42c0f None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:20:57.545278 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 44/87] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 24 06:20:57 2022] GET /identity/v3/projects/df345274c3b840b486c303c6b301052e => generated 298 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:20:57.546019 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1850a452-4004-4df2-a8d5-158a60e42c0f None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:20:57.649528 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a679467-8ee7-445b-a9c0-b4ea4c920aaf None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:20:57.660579 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1850a452-4004-4df2-a8d5-158a60e42c0f None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:20:57.686258 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e033e63f-2e23-47a2-8e96-af51ce1354a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:20:57.686901 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e033e63f-2e23-47a2-8e96-af51ce1354a2 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:57.687187 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e033e63f-2e23-47a2-8e96-af51ce1354a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:57.687466 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e033e63f-2e23-47a2-8e96-af51ce1354a2 None admin] PATH_INFO: `/v3/projects/df345274c3b840b486c303c6b301052e` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:57.689279 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e033e63f-2e23-47a2-8e96-af51ce1354a2 None admin] RBAC: Authorizing `identity:get_project(project_id=df345274c3b840b486c303c6b301052e)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:20:57.690732 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e033e63f-2e23-47a2-8e96-af51ce1354a2 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:20:57.692392 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 44/88] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 24 06:20:57 2022] GET /identity/v3/projects/df345274c3b840b486c303c6b301052e => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:20:57.693174 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e033e63f-2e23-47a2-8e96-af51ce1354a2 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:20:57.695972 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be718cbc-a1e4-4996-b2c1-07281e2154b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:20:57.696704 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be718cbc-a1e4-4996-b2c1-07281e2154b4 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:57.696981 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be718cbc-a1e4-4996-b2c1-07281e2154b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:57.697283 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be718cbc-a1e4-4996-b2c1-07281e2154b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:57.712382 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:20:57.758480 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:20:57.764760 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be718cbc-a1e4-4996-b2c1-07281e2154b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:20:57.766424 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be718cbc-a1e4-4996-b2c1-07281e2154b4 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:20:57.794374 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e033e63f-2e23-47a2-8e96-af51ce1354a2 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:20:57.828805 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e96d81c-624c-46a8-b972-854b9ec4453d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:20:57.829460 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e96d81c-624c-46a8-b972-854b9ec4453d None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:57.829748 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e96d81c-624c-46a8-b972-854b9ec4453d None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:57.830036 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e96d81c-624c-46a8-b972-854b9ec4453d None admin] PATH_INFO: `/v3/projects/df345274c3b840b486c303c6b301052e` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:57.831168 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e96d81c-624c-46a8-b972-854b9ec4453d None admin] RBAC: Authorizing `identity:get_project(project_id=df345274c3b840b486c303c6b301052e)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:20:57.832614 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e96d81c-624c-46a8-b972-854b9ec4453d None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:20:57.834291 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 45/89] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 24 06:20:57 2022] GET /identity/v3/projects/df345274c3b840b486c303c6b301052e => generated 298 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:20:57.835064 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e96d81c-624c-46a8-b972-854b9ec4453d None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:20:57.865521 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 45/90] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 24 06:20:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 06:20:57.866310 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be718cbc-a1e4-4996-b2c1-07281e2154b4 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:20:57.926357 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:20:57.940843 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e96d81c-624c-46a8-b972-854b9ec4453d None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:20:57.959955 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:20:57.965831 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be718cbc-a1e4-4996-b2c1-07281e2154b4 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:20:57.975288 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a59ebd5-8090-46a6-aada-cf26ca28a5e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:20:57.975906 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a59ebd5-8090-46a6-aada-cf26ca28a5e2 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:57.976180 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a59ebd5-8090-46a6-aada-cf26ca28a5e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:57.976453 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a59ebd5-8090-46a6-aada-cf26ca28a5e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:57.977210 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a59ebd5-8090-46a6-aada-cf26ca28a5e2 None admin] RBAC: Adding query filter params (role.id=e1f9b167c51c4a388cbc9594db8a465d, scope.project.id=df345274c3b840b486c303c6b301052e, user.id=b751994501284709b7f528880544d3b4) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:20:57.977518 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a59ebd5-8090-46a6-aada-cf26ca28a5e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:20:57.979001 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a59ebd5-8090-46a6-aada-cf26ca28a5e2 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:20:57.984417 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 46/91] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 24 06:20:57 2022] GET /identity/v3/role_assignments?user.id=b751994501284709b7f528880544d3b4&scope.project.id=df345274c3b840b486c303c6b301052e&role.id=e1f9b167c51c4a388cbc9594db8a465d => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:20:57.985936 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a59ebd5-8090-46a6-aada-cf26ca28a5e2 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:20:58.002883 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506a3e25-cbd1-46db-ab59-7563f3e5f892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:20:58.003548 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506a3e25-cbd1-46db-ab59-7563f3e5f892 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:58.003829 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506a3e25-cbd1-46db-ab59-7563f3e5f892 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:58.004348 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506a3e25-cbd1-46db-ab59-7563f3e5f892 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:58.005600 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506a3e25-cbd1-46db-ab59-7563f3e5f892 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:20:58.007162 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506a3e25-cbd1-46db-ab59-7563f3e5f892 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:20:58.009347 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 46/92] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:20:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:20:58.014970 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506a3e25-cbd1-46db-ab59-7563f3e5f892 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:20:58.106585 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a59ebd5-8090-46a6-aada-cf26ca28a5e2 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:20:58.141731 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506a3e25-cbd1-46db-ab59-7563f3e5f892 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:20:58.151543 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f98826-a11a-4f8b-a8c1-bd2ab5569269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:20:58.152176 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f98826-a11a-4f8b-a8c1-bd2ab5569269 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:58.152461 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f98826-a11a-4f8b-a8c1-bd2ab5569269 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:58.152736 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f98826-a11a-4f8b-a8c1-bd2ab5569269 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:58.153761 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f98826-a11a-4f8b-a8c1-bd2ab5569269 None admin] RBAC: Adding query filter params (role.id=b71c87e12dc1452ea3c99a7bf28585ca, scope.project.id=df345274c3b840b486c303c6b301052e, user.id=b751994501284709b7f528880544d3b4) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:20:58.154210 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f98826-a11a-4f8b-a8c1-bd2ab5569269 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:20:58.155548 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f98826-a11a-4f8b-a8c1-bd2ab5569269 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:20:58.169228 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 47/93] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 24 06:20:57 2022] GET /identity/v3/role_assignments?user.id=b751994501284709b7f528880544d3b4&scope.project.id=df345274c3b840b486c303c6b301052e&role.id=b71c87e12dc1452ea3c99a7bf28585ca => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:20:58.169228 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f98826-a11a-4f8b-a8c1-bd2ab5569269 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:20:58.190802 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f637bd-519f-4f7b-b084-4355397bbf58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:20:58.191230 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:20:58.191536 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f637bd-519f-4f7b-b084-4355397bbf58 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:58.192328 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f637bd-519f-4f7b-b084-4355397bbf58 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:58.192614 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f637bd-519f-4f7b-b084-4355397bbf58 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:58.193706 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f637bd-519f-4f7b-b084-4355397bbf58 None admin] RBAC: Adding query filter params (role.id=486144ad1bcd417e86acdbc1d14327bf, scope.project.id=df345274c3b840b486c303c6b301052e, user.id=029600e45e6b473db6751098bdba45e2) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:20:58.194023 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f637bd-519f-4f7b-b084-4355397bbf58 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:20:58.196246 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f637bd-519f-4f7b-b084-4355397bbf58 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:20:58.208794 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 47/94] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 24 06:20:58 2022] GET /identity/v3/role_assignments?user.id=029600e45e6b473db6751098bdba45e2&scope.project.id=df345274c3b840b486c303c6b301052e&role.id=486144ad1bcd417e86acdbc1d14327bf => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:20:58.210393 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f637bd-519f-4f7b-b084-4355397bbf58 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:20:58.252183 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:20:58.296941 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f98826-a11a-4f8b-a8c1-bd2ab5569269 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:20:58.339871 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f5c238-890f-4ed6-a1e5-a5c947cba824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:20:58.340562 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f5c238-890f-4ed6-a1e5-a5c947cba824 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:58.340853 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f5c238-890f-4ed6-a1e5-a5c947cba824 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:58.341136 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f5c238-890f-4ed6-a1e5-a5c947cba824 None admin] PATH_INFO: `/v3/projects/58c030ee4f10414aa5f379a1c9cab0be` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:58.342637 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f5c238-890f-4ed6-a1e5-a5c947cba824 None admin] RBAC: Authorizing `identity:get_project(project_id=58c030ee4f10414aa5f379a1c9cab0be)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:20:58.345019 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f5c238-890f-4ed6-a1e5-a5c947cba824 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:20:58.347131 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 48/95] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 24 06:20:58 2022] GET /identity/v3/projects/58c030ee4f10414aa5f379a1c9cab0be => generated 312 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:20:58.352688 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f637bd-519f-4f7b-b084-4355397bbf58 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:20:58.353405 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f5c238-890f-4ed6-a1e5-a5c947cba824 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:20:58.400390 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bd5533-8e18-4610-8abd-cf10c9b6764d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:20:58.401036 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bd5533-8e18-4610-8abd-cf10c9b6764d None admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:58.401345 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bd5533-8e18-4610-8abd-cf10c9b6764d None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:58.401625 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bd5533-8e18-4610-8abd-cf10c9b6764d None admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:58.402283 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bd5533-8e18-4610-8abd-cf10c9b6764d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:20:58.404039 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bd5533-8e18-4610-8abd-cf10c9b6764d None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:20:58.464177 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:20:58.487156 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f5c238-890f-4ed6-a1e5-a5c947cba824 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:20:58.527587 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 48/96] 10.222.0.47 () {60 vars in 1132 bytes} [Wed Aug 24 06:20:58 2022] POST /identity/v3/users => generated 289 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:20:58.530894 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d94a3d-0525-4b32-bfdf-30e9e33b8494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:20:58.532145 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d94a3d-0525-4b32-bfdf-30e9e33b8494 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:20:58.532592 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d94a3d-0525-4b32-bfdf-30e9e33b8494 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:20:58.533099 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d94a3d-0525-4b32-bfdf-30e9e33b8494 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:20:58.533918 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d94a3d-0525-4b32-bfdf-30e9e33b8494 None admin] RBAC: Adding query filter params (role.id=e1f9b167c51c4a388cbc9594db8a465d, scope.project.id=58c030ee4f10414aa5f379a1c9cab0be, user.id=b751994501284709b7f528880544d3b4) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:20:58.534233 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d94a3d-0525-4b32-bfdf-30e9e33b8494 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:20:58.535745 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d94a3d-0525-4b32-bfdf-30e9e33b8494 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:20:58.542307 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 49/97] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 24 06:20:58 2022] GET /identity/v3/role_assignments?user.id=b751994501284709b7f528880544d3b4&scope.project.id=58c030ee4f10414aa5f379a1c9cab0be&role.id=e1f9b167c51c4a388cbc9594db8a465d => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:21:00.360916 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5c76db-f697-4985-b2f2-cda91c084907 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:00.361510 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5c76db-f697-4985-b2f2-cda91c084907 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:00.361753 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5c76db-f697-4985-b2f2-cda91c084907 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:00.363565 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 49/98] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:21:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:21:00.368694 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b2d69f-62f4-4c7f-9bdd-eea47e3a4480 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:00.369147 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b2d69f-62f4-4c7f-9bdd-eea47e3a4480 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:00.369487 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b2d69f-62f4-4c7f-9bdd-eea47e3a4480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:00.394291 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-e9b2d69f-62f4-4c7f-9bdd-eea47e3a4480 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:21:00.395541 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-e9b2d69f-62f4-4c7f-9bdd-eea47e3a4480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:21:00.483417 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 50/99] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:21:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:21:00.670981 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db49023-04ad-46f0-b60f-b4495f709cf0 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:00.671311 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db49023-04ad-46f0-b60f-b4495f709cf0 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:00.671546 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db49023-04ad-46f0-b60f-b4495f709cf0 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:00.674252 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 50/100] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:21:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:21:00.679451 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefab85c-bbcd-49c2-985b-cef1adf93946 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:00.679750 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefab85c-bbcd-49c2-985b-cef1adf93946 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:00.680037 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefab85c-bbcd-49c2-985b-cef1adf93946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:00.705018 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-aefab85c-bbcd-49c2-985b-cef1adf93946 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:21:00.706309 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-aefab85c-bbcd-49c2-985b-cef1adf93946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:21:00.801895 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 51/101] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:21:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:21:00.873634 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca185c73-a680-40c7-b3e6-239b14147698 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:00.873926 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca185c73-a680-40c7-b3e6-239b14147698 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:00.874126 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca185c73-a680-40c7-b3e6-239b14147698 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:00.876582 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 51/102] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:21:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:21:00.886909 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e402cc4-7545-4c91-8633-8bed6d300ed9 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:00.887289 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e402cc4-7545-4c91-8633-8bed6d300ed9 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:00.887607 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e402cc4-7545-4c91-8633-8bed6d300ed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:00.914573 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:00.922852 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-4e402cc4-7545-4c91-8633-8bed6d300ed9 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:21:00.927329 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-4e402cc4-7545-4c91-8633-8bed6d300ed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:21:01.015082 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 52/103] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:21:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:21:01.067043 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab84ad69-88ec-4dc5-bfff-0834d43021b4 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:01.067467 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab84ad69-88ec-4dc5-bfff-0834d43021b4 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:01.067643 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab84ad69-88ec-4dc5-bfff-0834d43021b4 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:01.068959 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 52/104] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:21:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:21:01.074403 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82306f81-af98-401a-8cf5-5dfaf3245330 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:01.074711 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82306f81-af98-401a-8cf5-5dfaf3245330 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:01.074966 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82306f81-af98-401a-8cf5-5dfaf3245330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:01.102179 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-82306f81-af98-401a-8cf5-5dfaf3245330 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:21:01.103399 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-82306f81-af98-401a-8cf5-5dfaf3245330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:21:01.129503 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab84ad69-88ec-4dc5-bfff-0834d43021b4 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:01.212232 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 53/105] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:21:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:21:01.234113 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82306f81-af98-401a-8cf5-5dfaf3245330 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:01.295570 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:01.346919 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab84ad69-88ec-4dc5-bfff-0834d43021b4 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:01.410038 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b88487e-a10c-4f3f-a297-93d703a02717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:01.410670 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b88487e-a10c-4f3f-a297-93d703a02717 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:01.411055 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b88487e-a10c-4f3f-a297-93d703a02717 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:01.411345 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b88487e-a10c-4f3f-a297-93d703a02717 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:01.411638 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:01.433110 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b88487e-a10c-4f3f-a297-93d703a02717 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:21:01.438959 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-6b88487e-a10c-4f3f-a297-93d703a02717 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 06:21:01.438959 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:21:01.438959 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:21:01.438959 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:21:01.438959 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:21:01.438959 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:21:01.438959 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:21:01.438959 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:21:01.438959 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:21:01.438959 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:21:01.438959 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:21:01.438959 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:21:01.438959 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:21:01.438959 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:21:01.438959 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:21:01.438959 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:21:01.439916 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:21:01.439916 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:21:01.439916 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:21:01.439916 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:21:01.439916 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:21:01.439916 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:21:01.439916 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:21:01.439916 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:21:01.439916 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:21:01.439916 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:21:01.439916 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:21:01.439916 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:21:01.439916 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:21:01.439916 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:21:01.439916 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:21:01.439916 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:21:01.440691 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:21:01.440691 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:21:01.440691 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:21:01.440691 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:21:01.440691 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:21:01.440691 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:21:01.440691 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:21:01.440691 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:21:01.440691 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:21:01.440691 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:21:01.440691 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 06:21:01.440691 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:21:01.440691 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 53/106] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 24 06:21:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:21:01.447973 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b88487e-a10c-4f3f-a297-93d703a02717 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:01.543436 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82306f81-af98-401a-8cf5-5dfaf3245330 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:01.628947 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b88487e-a10c-4f3f-a297-93d703a02717 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:01.631544 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca90f974-e966-47cc-bbf8-a9eda7f6e05b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:01.632204 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca90f974-e966-47cc-bbf8-a9eda7f6e05b None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:01.632484 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca90f974-e966-47cc-bbf8-a9eda7f6e05b None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:01.632754 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca90f974-e966-47cc-bbf8-a9eda7f6e05b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:01.652878 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca90f974-e966-47cc-bbf8-a9eda7f6e05b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:21:01.660742 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-ca90f974-e966-47cc-bbf8-a9eda7f6e05b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:21:01.660742 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:21:01.660742 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:21:01.660742 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:21:01.660742 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:21:01.660742 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:21:01.660742 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:21:01.660742 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:21:01.660742 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:21:01.660742 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:21:01.660742 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:21:01.660742 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:21:01.660742 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:21:01.660742 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:21:01.660742 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:21:01.660742 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:21:01.661713 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:21:01.661713 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:21:01.661713 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:21:01.661713 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:21:01.661713 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:21:01.661713 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:21:01.661713 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:21:01.661713 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:21:01.661713 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:21:01.661713 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:21:01.661713 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:21:01.661713 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:21:01.661713 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:21:01.661713 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:21:01.661713 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:21:01.661713 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:21:01.662479 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:21:01.662479 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:21:01.662479 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:21:01.662479 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:21:01.662479 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:21:01.662479 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:21:01.662479 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:21:01.662479 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:21:01.662479 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:21:01.662479 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:21:01.662479 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:21:01.662479 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:21:01.662479 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 54/107] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Aug 24 06:21:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 429 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:21:01.663758 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca90f974-e966-47cc-bbf8-a9eda7f6e05b None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:01.679132 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8dfba8-7ae9-441a-9f53-534b45e6b308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:01.679927 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8dfba8-7ae9-441a-9f53-534b45e6b308 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:01.680217 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8dfba8-7ae9-441a-9f53-534b45e6b308 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:01.680497 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8dfba8-7ae9-441a-9f53-534b45e6b308 None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:01.681741 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8dfba8-7ae9-441a-9f53-534b45e6b308 None admin] RBAC: Adding query filter params (name=member) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:01.682191 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8dfba8-7ae9-441a-9f53-534b45e6b308 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:01.683774 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8dfba8-7ae9-441a-9f53-534b45e6b308 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:01.701282 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 54/108] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Aug 24 06:21:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:01.702255 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8dfba8-7ae9-441a-9f53-534b45e6b308 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:01.714730 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:01.782581 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca90f974-e966-47cc-bbf8-a9eda7f6e05b None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:01.810950 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:01.821007 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2197c0d1-c2c6-40d5-8fd0-05f141f51d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:01.825448 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2197c0d1-c2c6-40d5-8fd0-05f141f51d88 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:01.825448 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2197c0d1-c2c6-40d5-8fd0-05f141f51d88 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:01.825448 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2197c0d1-c2c6-40d5-8fd0-05f141f51d88 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:01.835567 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2197c0d1-c2c6-40d5-8fd0-05f141f51d88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:21:01.835976 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8dfba8-7ae9-441a-9f53-534b45e6b308 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:01.841240 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-2197c0d1-c2c6-40d5-8fd0-05f141f51d88 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 06:21:01.841240 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:21:01.841240 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:21:01.841240 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:21:01.841240 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:21:01.841240 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:21:01.841240 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:21:01.841240 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:21:01.841240 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:21:01.841240 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:21:01.841240 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:21:01.841240 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:21:01.841240 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:21:01.841240 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:21:01.841240 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:21:01.841240 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:21:01.842275 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:21:01.842275 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:21:01.842275 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:21:01.842275 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:21:01.842275 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:21:01.842275 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:21:01.842275 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:21:01.842275 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:21:01.842275 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:21:01.842275 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:21:01.842275 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:21:01.842275 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:21:01.842275 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:21:01.842275 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:21:01.842275 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:21:01.842275 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:21:01.843164 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:21:01.843164 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:21:01.843164 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:21:01.843164 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:21:01.843164 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:21:01.843164 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:21:01.843164 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:21:01.843164 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:21:01.843164 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:21:01.843164 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:21:01.843164 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 06:21:01.843164 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:21:01.845948 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 55/109] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Aug 24 06:21:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:21:01.847001 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2197c0d1-c2c6-40d5-8fd0-05f141f51d88 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:01.883847 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5a13ce-c420-4d13-9f3c-a091a4c9a0dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:01.884995 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5a13ce-c420-4d13-9f3c-a091a4c9a0dd None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:01.885309 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5a13ce-c420-4d13-9f3c-a091a4c9a0dd None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:01.885593 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5a13ce-c420-4d13-9f3c-a091a4c9a0dd None admin] PATH_INFO: `/v3/roles/member` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:01.898671 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5a13ce-c420-4d13-9f3c-a091a4c9a0dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:21:01.905516 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-5a5a13ce-c420-4d13-9f3c-a091a4c9a0dd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 06:21:01.905516 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:21:01.905516 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:21:01.905516 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:21:01.905516 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:21:01.905516 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:21:01.905516 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:21:01.905516 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:21:01.905516 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:21:01.905516 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:21:01.905516 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:21:01.905516 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:21:01.905516 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:21:01.905516 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:21:01.905516 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:21:01.905516 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:21:01.907251 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:21:01.907251 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:21:01.907251 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:21:01.907251 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:21:01.907251 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:21:01.907251 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:21:01.907251 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:21:01.907251 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:21:01.907251 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:21:01.907251 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:21:01.907251 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:21:01.907251 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:21:01.907251 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:21:01.907251 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:21:01.907251 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:21:01.907251 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:21:01.908852 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:21:01.908852 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:21:01.908852 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:21:01.908852 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:21:01.908852 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:21:01.908852 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:21:01.908852 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:21:01.908852 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:21:01.908852 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:21:01.908852 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:21:01.908852 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 06:21:01.908852 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:21:01.908852 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 55/110] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 24 06:21:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:21:01.908852 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5a13ce-c420-4d13-9f3c-a091a4c9a0dd None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:01.982335 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:02.024124 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5a13ce-c420-4d13-9f3c-a091a4c9a0dd None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:02.054516 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2197c0d1-c2c6-40d5-8fd0-05f141f51d88 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:02.063906 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d669ad-161c-433c-a5d8-89573c41dd0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:02.064788 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d669ad-161c-433c-a5d8-89573c41dd0c None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:02.065013 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d669ad-161c-433c-a5d8-89573c41dd0c None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:02.065322 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d669ad-161c-433c-a5d8-89573c41dd0c None admin] PATH_INFO: `/v3/users/b751994501284709b7f528880544d3b4` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:02.066524 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d669ad-161c-433c-a5d8-89573c41dd0c None admin] RBAC: Authorizing `identity:get_user(user_id=b751994501284709b7f528880544d3b4)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:02.067977 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d669ad-161c-433c-a5d8-89573c41dd0c None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:02.074274 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 56/111] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 24 06:21:01 2022] GET /identity/v3/users/b751994501284709b7f528880544d3b4 => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:02.074274 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d669ad-161c-433c-a5d8-89573c41dd0c None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:02.103848 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdfc0dc-fecf-4b96-8867-4f0879720de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:02.104470 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdfc0dc-fecf-4b96-8867-4f0879720de5 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:02.104749 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdfc0dc-fecf-4b96-8867-4f0879720de5 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:02.105036 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdfc0dc-fecf-4b96-8867-4f0879720de5 None admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:02.105798 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdfc0dc-fecf-4b96-8867-4f0879720de5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:02.106224 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdfc0dc-fecf-4b96-8867-4f0879720de5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:02.107731 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdfc0dc-fecf-4b96-8867-4f0879720de5 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:02.128866 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 56/112] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 24 06:21:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:02.128866 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdfc0dc-fecf-4b96-8867-4f0879720de5 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:02.186202 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:02.207361 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d669ad-161c-433c-a5d8-89573c41dd0c None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:02.245571 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233bef01-fbe4-4b49-8f30-8a5886857cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:02.246172 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233bef01-fbe4-4b49-8f30-8a5886857cb9 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:02.246436 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233bef01-fbe4-4b49-8f30-8a5886857cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:02.246720 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233bef01-fbe4-4b49-8f30-8a5886857cb9 None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:02.247390 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233bef01-fbe4-4b49-8f30-8a5886857cb9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:02.247695 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233bef01-fbe4-4b49-8f30-8a5886857cb9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:02.249088 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233bef01-fbe4-4b49-8f30-8a5886857cb9 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:02.251737 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:02.252020 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdfc0dc-fecf-4b96-8867-4f0879720de5 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:02.261224 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 57/113] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 24 06:21:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:02.262307 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673bf071-c1e2-4112-b0ad-30fae1710eb6 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:02.272915 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673bf071-c1e2-4112-b0ad-30fae1710eb6 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:02.272915 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673bf071-c1e2-4112-b0ad-30fae1710eb6 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:02.272915 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 58/114] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:21:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:21:02.272915 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673bf071-c1e2-4112-b0ad-30fae1710eb6 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:02.293205 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38692311-a36b-4a5c-92b5-a36fc45aa706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:02.293823 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38692311-a36b-4a5c-92b5-a36fc45aa706 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:02.294107 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38692311-a36b-4a5c-92b5-a36fc45aa706 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:02.294382 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38692311-a36b-4a5c-92b5-a36fc45aa706 None admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:02.295053 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38692311-a36b-4a5c-92b5-a36fc45aa706 None admin] RBAC: Adding query filter params (name=member) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:02.295357 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38692311-a36b-4a5c-92b5-a36fc45aa706 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:02.297310 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38692311-a36b-4a5c-92b5-a36fc45aa706 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:02.306176 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 57/115] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Aug 24 06:21:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:02.307008 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38692311-a36b-4a5c-92b5-a36fc45aa706 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:02.383167 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673bf071-c1e2-4112-b0ad-30fae1710eb6 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:02.412787 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38692311-a36b-4a5c-92b5-a36fc45aa706 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:02.419633 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dadfe7-9de2-4c2c-b7eb-993ef2290305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:02.420281 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dadfe7-9de2-4c2c-b7eb-993ef2290305 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:02.420569 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dadfe7-9de2-4c2c-b7eb-993ef2290305 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:02.420843 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dadfe7-9de2-4c2c-b7eb-993ef2290305 None admin] PATH_INFO: `/v3/projects/df345274c3b840b486c303c6b301052e` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:02.422860 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dadfe7-9de2-4c2c-b7eb-993ef2290305 None admin] RBAC: Authorizing `identity:get_project(project_id=df345274c3b840b486c303c6b301052e)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:02.424342 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dadfe7-9de2-4c2c-b7eb-993ef2290305 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:02.424868 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:02.426067 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 59/116] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 24 06:21:02 2022] GET /identity/v3/projects/df345274c3b840b486c303c6b301052e => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:02.430506 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9a1dcc-3b7c-4b4a-ae89-f3ee00ea3934 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:02.430796 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9a1dcc-3b7c-4b4a-ae89-f3ee00ea3934 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:02.431356 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9a1dcc-3b7c-4b4a-ae89-f3ee00ea3934 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:02.432692 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 60/117] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:21:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:21:02.433453 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9a1dcc-3b7c-4b4a-ae89-f3ee00ea3934 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:02.457498 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ec03ea-40b8-44a1-895f-20920bdd5578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:02.458159 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ec03ea-40b8-44a1-895f-20920bdd5578 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:02.458803 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ec03ea-40b8-44a1-895f-20920bdd5578 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:02.459146 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ec03ea-40b8-44a1-895f-20920bdd5578 None admin] PATH_INFO: `/v3/users/029600e45e6b473db6751098bdba45e2` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:02.462424 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ec03ea-40b8-44a1-895f-20920bdd5578 None admin] RBAC: Authorizing `identity:get_user(user_id=029600e45e6b473db6751098bdba45e2)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:02.463900 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ec03ea-40b8-44a1-895f-20920bdd5578 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:02.467304 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 58/118] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 24 06:21:02 2022] GET /identity/v3/users/029600e45e6b473db6751098bdba45e2 => generated 247 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:02.470718 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e48e77-3d9f-4a74-ac5c-cefaa80629fb None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:02.471026 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e48e77-3d9f-4a74-ac5c-cefaa80629fb None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:02.471363 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e48e77-3d9f-4a74-ac5c-cefaa80629fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:02.497971 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-19e48e77-3d9f-4a74-ac5c-cefaa80629fb None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:21:02.499093 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-19e48e77-3d9f-4a74-ac5c-cefaa80629fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:21:02.502109 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:02.543530 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9a1dcc-3b7c-4b4a-ae89-f3ee00ea3934 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:02.567306 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 59/119] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:21:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:21:02.570892 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e48e77-3d9f-4a74-ac5c-cefaa80629fb None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:02.582091 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55830fc-fdab-4b32-96c0-4a3e85357ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:02.582606 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55830fc-fdab-4b32-96c0-4a3e85357ad4 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:02.582877 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55830fc-fdab-4b32-96c0-4a3e85357ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:02.583152 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55830fc-fdab-4b32-96c0-4a3e85357ad4 None admin] PATH_INFO: `/v3/users/b751994501284709b7f528880544d3b4` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:02.584448 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55830fc-fdab-4b32-96c0-4a3e85357ad4 None admin] RBAC: Authorizing `identity:get_user(user_id=b751994501284709b7f528880544d3b4)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:02.585910 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55830fc-fdab-4b32-96c0-4a3e85357ad4 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:02.587756 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 61/120] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 24 06:21:02 2022] GET /identity/v3/users/b751994501284709b7f528880544d3b4 => generated 275 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:02.593979 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55830fc-fdab-4b32-96c0-4a3e85357ad4 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:02.689441 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e48e77-3d9f-4a74-ac5c-cefaa80629fb None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:02.699691 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:02.723640 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55830fc-fdab-4b32-96c0-4a3e85357ad4 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:02.735967 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcafe7c-aff7-4526-bc28-7ac00a47346b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:02.736954 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcafe7c-aff7-4526-bc28-7ac00a47346b None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:02.737281 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcafe7c-aff7-4526-bc28-7ac00a47346b None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:02.737601 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcafe7c-aff7-4526-bc28-7ac00a47346b None admin] PATH_INFO: `/v3/users/b751994501284709b7f528880544d3b4` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:02.738824 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcafe7c-aff7-4526-bc28-7ac00a47346b None admin] RBAC: Authorizing `identity:get_user(user_id=b751994501284709b7f528880544d3b4)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:02.740292 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcafe7c-aff7-4526-bc28-7ac00a47346b None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:02.741960 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 60/121] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 24 06:21:02 2022] GET /identity/v3/users/b751994501284709b7f528880544d3b4 => generated 275 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:02.747891 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef3d17-d478-400a-9b79-7ba2e53ff88b None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:02.748240 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef3d17-d478-400a-9b79-7ba2e53ff88b None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:02.748468 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef3d17-d478-400a-9b79-7ba2e53ff88b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:02.759708 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:02.766532 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276491b4-52f3-4531-8258-3663bda65622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:02.767377 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276491b4-52f3-4531-8258-3663bda65622 None admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:02.767658 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276491b4-52f3-4531-8258-3663bda65622 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:02.767932 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-93ef3d17-d478-400a-9b79-7ba2e53ff88b None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:21:02.768022 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276491b4-52f3-4531-8258-3663bda65622 None admin] PATH_INFO: `/v3/projects/df345274c3b840b486c303c6b301052e/users/b751994501284709b7f528880544d3b4/roles/e1f9b167c51c4a388cbc9594db8a465d` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:02.776220 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-93ef3d17-d478-400a-9b79-7ba2e53ff88b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:21:02.776351 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276491b4-52f3-4531-8258-3663bda65622 None admin] RBAC: Authorizing `identity:create_grant(project_id=df345274c3b840b486c303c6b301052e, user_id=b751994501284709b7f528880544d3b4, role_id=e1f9b167c51c4a388cbc9594db8a465d)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:02.776351 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276491b4-52f3-4531-8258-3663bda65622 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:02.843423 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 61/122] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:21:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:21:02.845805 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ef3d17-d478-400a-9b79-7ba2e53ff88b None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:02.869512 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 62/123] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Aug 24 06:21:02 2022] PUT /identity/v3/projects/df345274c3b840b486c303c6b301052e/users/b751994501284709b7f528880544d3b4/roles/e1f9b167c51c4a388cbc9594db8a465d => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:21:02.882204 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276491b4-52f3-4531-8258-3663bda65622 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:02.983509 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ef3d17-d478-400a-9b79-7ba2e53ff88b None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:02.993033 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:03.014506 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276491b4-52f3-4531-8258-3663bda65622 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:03.029019 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e767542-fff6-4215-abe2-87744034ecdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:03.030079 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e767542-fff6-4215-abe2-87744034ecdb None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:03.030628 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e767542-fff6-4215-abe2-87744034ecdb None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:03.031659 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e767542-fff6-4215-abe2-87744034ecdb None admin] PATH_INFO: `/v3/projects/df345274c3b840b486c303c6b301052e` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:03.033603 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e767542-fff6-4215-abe2-87744034ecdb None admin] RBAC: Authorizing `identity:get_project(project_id=df345274c3b840b486c303c6b301052e)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:03.035282 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e767542-fff6-4215-abe2-87744034ecdb None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:03.037361 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 62/124] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 24 06:21:02 2022] GET /identity/v3/projects/df345274c3b840b486c303c6b301052e => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:03.037919 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e767542-fff6-4215-abe2-87744034ecdb None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:03.058156 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe72df5f-ac34-4977-84e2-da05d7cdca7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:03.058813 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe72df5f-ac34-4977-84e2-da05d7cdca7d None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:03.059092 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe72df5f-ac34-4977-84e2-da05d7cdca7d None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:03.059371 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe72df5f-ac34-4977-84e2-da05d7cdca7d None admin] PATH_INFO: `/v3/projects/df345274c3b840b486c303c6b301052e` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:03.060768 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe72df5f-ac34-4977-84e2-da05d7cdca7d None admin] RBAC: Authorizing `identity:get_project(project_id=df345274c3b840b486c303c6b301052e)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:03.062412 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe72df5f-ac34-4977-84e2-da05d7cdca7d None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:03.064373 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 63/125] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 24 06:21:02 2022] GET /identity/v3/projects/df345274c3b840b486c303c6b301052e => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:03.065726 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe72df5f-ac34-4977-84e2-da05d7cdca7d None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:03.142653 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:03.186672 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e767542-fff6-4215-abe2-87744034ecdb None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:03.187775 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe72df5f-ac34-4977-84e2-da05d7cdca7d None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:03.228665 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcb10ec-e52a-47bd-85bf-bf09c1e622d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:03.229116 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1786994-e40b-452a-a4f3-1057295dd93a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:03.229353 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcb10ec-e52a-47bd-85bf-bf09c1e622d0 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:03.229619 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcb10ec-e52a-47bd-85bf-bf09c1e622d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:03.229786 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1786994-e40b-452a-a4f3-1057295dd93a None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:03.229891 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcb10ec-e52a-47bd-85bf-bf09c1e622d0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:03.230075 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1786994-e40b-452a-a4f3-1057295dd93a None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:03.230355 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1786994-e40b-452a-a4f3-1057295dd93a None admin] PATH_INFO: `/v3/projects/58c030ee4f10414aa5f379a1c9cab0be` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:03.231504 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1786994-e40b-452a-a4f3-1057295dd93a None admin] RBAC: Authorizing `identity:get_project(project_id=58c030ee4f10414aa5f379a1c9cab0be)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:03.233536 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1786994-e40b-452a-a4f3-1057295dd93a None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:03.241499 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 63/126] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 24 06:21:03 2022] GET /identity/v3/projects/58c030ee4f10414aa5f379a1c9cab0be => generated 312 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:03.241499 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1786994-e40b-452a-a4f3-1057295dd93a None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:03.247699 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcb10ec-e52a-47bd-85bf-bf09c1e622d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:21:03.254241 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-ffcb10ec-e52a-47bd-85bf-bf09c1e622d0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 06:21:03.254241 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:21:03.254241 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:21:03.254241 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:21:03.254241 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:21:03.254241 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:21:03.254241 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:21:03.254241 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:21:03.254241 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:21:03.254241 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:21:03.254241 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:21:03.254241 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:21:03.254241 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:21:03.254241 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:21:03.254241 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:21:03.254241 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:21:03.255388 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:21:03.255388 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:21:03.255388 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:21:03.255388 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:21:03.255388 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:21:03.255388 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:21:03.255388 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:21:03.255388 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:21:03.255388 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:21:03.255388 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:21:03.255388 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:21:03.255388 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:21:03.255388 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:21:03.255388 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:21:03.255388 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:21:03.255388 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:21:03.256343 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:21:03.256343 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:21:03.256343 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:21:03.256343 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:21:03.256343 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:21:03.256343 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:21:03.256343 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:21:03.256343 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:21:03.256343 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:21:03.256343 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:21:03.256343 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 06:21:03.256343 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:21:03.256343 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 64/127] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 24 06:21:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:21:03.257118 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcb10ec-e52a-47bd-85bf-bf09c1e622d0 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:03.299413 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:03.370625 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcb10ec-e52a-47bd-85bf-bf09c1e622d0 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:03.383106 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1786994-e40b-452a-a4f3-1057295dd93a None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:03.393519 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:03.410566 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad587a71-0b57-4fb5-9c1a-7cb0f9895a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:03.411249 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad587a71-0b57-4fb5-9c1a-7cb0f9895a53 None admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:03.411602 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad587a71-0b57-4fb5-9c1a-7cb0f9895a53 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:03.411817 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad587a71-0b57-4fb5-9c1a-7cb0f9895a53 None admin] PATH_INFO: `/v3/projects/df345274c3b840b486c303c6b301052e/users/b751994501284709b7f528880544d3b4/roles/b71c87e12dc1452ea3c99a7bf28585ca` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:03.414042 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad587a71-0b57-4fb5-9c1a-7cb0f9895a53 None admin] RBAC: Authorizing `identity:create_grant(project_id=df345274c3b840b486c303c6b301052e, user_id=b751994501284709b7f528880544d3b4, role_id=b71c87e12dc1452ea3c99a7bf28585ca)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:03.415501 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad587a71-0b57-4fb5-9c1a-7cb0f9895a53 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:03.422631 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe72c76b-dd82-4489-be16-f10291f90ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:03.425030 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe72c76b-dd82-4489-be16-f10291f90ef9 None admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:03.425030 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe72c76b-dd82-4489-be16-f10291f90ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:03.425030 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe72c76b-dd82-4489-be16-f10291f90ef9 None admin] PATH_INFO: `/v3/projects/df345274c3b840b486c303c6b301052e/users/029600e45e6b473db6751098bdba45e2/roles/486144ad1bcd417e86acdbc1d14327bf` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:03.425860 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe72c76b-dd82-4489-be16-f10291f90ef9 None admin] RBAC: Authorizing `identity:create_grant(project_id=df345274c3b840b486c303c6b301052e, user_id=029600e45e6b473db6751098bdba45e2, role_id=486144ad1bcd417e86acdbc1d14327bf)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:03.427334 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe72c76b-dd82-4489-be16-f10291f90ef9 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:03.508850 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 65/128] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Aug 24 06:21:03 2022] PUT /identity/v3/projects/df345274c3b840b486c303c6b301052e/users/b751994501284709b7f528880544d3b4/roles/b71c87e12dc1452ea3c99a7bf28585ca => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:21:03.517731 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad587a71-0b57-4fb5-9c1a-7cb0f9895a53 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:03.541509 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 64/129] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Aug 24 06:21:03 2022] PUT /identity/v3/projects/df345274c3b840b486c303c6b301052e/users/029600e45e6b473db6751098bdba45e2/roles/486144ad1bcd417e86acdbc1d14327bf => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:21:03.543287 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe72c76b-dd82-4489-be16-f10291f90ef9 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:03.707303 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad587a71-0b57-4fb5-9c1a-7cb0f9895a53 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:03.709383 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe72c76b-dd82-4489-be16-f10291f90ef9 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:03.713547 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:03.769916 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d93511-7cf9-4898-a746-32cdf2b3046d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:03.774135 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d93511-7cf9-4898-a746-32cdf2b3046d None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:03.774135 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d93511-7cf9-4898-a746-32cdf2b3046d None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:03.774135 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d93511-7cf9-4898-a746-32cdf2b3046d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:03.779421 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d93511-7cf9-4898-a746-32cdf2b3046d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:03.780984 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d93511-7cf9-4898-a746-32cdf2b3046d None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:03.782386 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff40793-280f-4d85-9973-f09b2fc1b314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:03.784142 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff40793-280f-4d85-9973-f09b2fc1b314 None admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:03.784142 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff40793-280f-4d85-9973-f09b2fc1b314 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:03.784142 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff40793-280f-4d85-9973-f09b2fc1b314 None admin] PATH_INFO: `/v3/projects/58c030ee4f10414aa5f379a1c9cab0be/users/b751994501284709b7f528880544d3b4/roles/e1f9b167c51c4a388cbc9594db8a465d` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:03.786598 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 66/130] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:21:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:03.787303 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d93511-7cf9-4898-a746-32cdf2b3046d None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:03.789516 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff40793-280f-4d85-9973-f09b2fc1b314 None admin] RBAC: Authorizing `identity:create_grant(project_id=58c030ee4f10414aa5f379a1c9cab0be, user_id=b751994501284709b7f528880544d3b4, role_id=e1f9b167c51c4a388cbc9594db8a465d)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:03.791100 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff40793-280f-4d85-9973-f09b2fc1b314 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:03.855900 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:03.860910 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 65/131] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Aug 24 06:21:03 2022] PUT /identity/v3/projects/58c030ee4f10414aa5f379a1c9cab0be/users/b751994501284709b7f528880544d3b4/roles/e1f9b167c51c4a388cbc9594db8a465d => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:21:03.862319 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff40793-280f-4d85-9973-f09b2fc1b314 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:03.983582 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d93511-7cf9-4898-a746-32cdf2b3046d None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:04.006050 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff40793-280f-4d85-9973-f09b2fc1b314 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:04.051114 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa70ec4d-4add-4d61-90fd-6e2a645fc372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:04.052174 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa70ec4d-4add-4d61-90fd-6e2a645fc372 None admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:04.052660 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa70ec4d-4add-4d61-90fd-6e2a645fc372 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:04.053146 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa70ec4d-4add-4d61-90fd-6e2a645fc372 None admin] PATH_INFO: `/v3/projects` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:04.054268 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa70ec4d-4add-4d61-90fd-6e2a645fc372 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:04.056331 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa70ec4d-4add-4d61-90fd-6e2a645fc372 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:04.086557 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b06af19-112b-468e-a377-30b643ca8f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:04.093408 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b06af19-112b-468e-a377-30b643ca8f45 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:04.093408 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b06af19-112b-468e-a377-30b643ca8f45 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:04.093408 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b06af19-112b-468e-a377-30b643ca8f45 None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:04.093408 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b06af19-112b-468e-a377-30b643ca8f45 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:04.093408 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b06af19-112b-468e-a377-30b643ca8f45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:04.093408 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b06af19-112b-468e-a377-30b643ca8f45 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:04.103184 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 67/132] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Aug 24 06:21:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:04.111813 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b06af19-112b-468e-a377-30b643ca8f45 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:04.158435 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 66/133] 10.222.0.47 () {60 vars in 1141 bytes} [Wed Aug 24 06:21:03 2022] POST /identity/v3/projects => generated 302 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:21:04.266778 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b06af19-112b-468e-a377-30b643ca8f45 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:04.279631 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:04.305994 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a058e86-b0c7-4317-9567-94d094185991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:04.306740 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a058e86-b0c7-4317-9567-94d094185991 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:04.307089 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a058e86-b0c7-4317-9567-94d094185991 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:04.307431 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a058e86-b0c7-4317-9567-94d094185991 None admin] PATH_INFO: `/v3/users/f77fd3d9bc3c45bcafb43f716824579a` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:04.324300 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a058e86-b0c7-4317-9567-94d094185991 None admin] RBAC: Authorizing `identity:get_user(user_id=f77fd3d9bc3c45bcafb43f716824579a)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:04.325963 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a058e86-b0c7-4317-9567-94d094185991 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:04.328613 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 68/134] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 24 06:21:04 2022] GET /identity/v3/users/f77fd3d9bc3c45bcafb43f716824579a => generated 289 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:04.334374 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa70ec4d-4add-4d61-90fd-6e2a645fc372 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:04.354526 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:04.466745 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa70ec4d-4add-4d61-90fd-6e2a645fc372 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:04.507405 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ef4fbd-bdde-4d71-8fca-3ca8fa4da49b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:04.508121 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ef4fbd-bdde-4d71-8fca-3ca8fa4da49b None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:04.508473 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ef4fbd-bdde-4d71-8fca-3ca8fa4da49b None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:04.508816 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ef4fbd-bdde-4d71-8fca-3ca8fa4da49b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:04.594505 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ef4fbd-bdde-4d71-8fca-3ca8fa4da49b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:04.596066 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ef4fbd-bdde-4d71-8fca-3ca8fa4da49b None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:04.612944 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:04.731092 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 67/135] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 24 06:21:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 06:21:04.738376 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a058e86-b0c7-4317-9567-94d094185991 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:04.866717 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a058e86-b0c7-4317-9567-94d094185991 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:04.913684 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0cb57f-27c7-4331-842b-638ebbfee532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:04.914386 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0cb57f-27c7-4331-842b-638ebbfee532 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:04.914671 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0cb57f-27c7-4331-842b-638ebbfee532 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:04.914993 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0cb57f-27c7-4331-842b-638ebbfee532 None admin] PATH_INFO: `/v3/projects/df345274c3b840b486c303c6b301052e` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:04.916187 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0cb57f-27c7-4331-842b-638ebbfee532 None admin] RBAC: Authorizing `identity:get_project(project_id=df345274c3b840b486c303c6b301052e)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:04.917682 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0cb57f-27c7-4331-842b-638ebbfee532 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:04.919321 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 69/136] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 24 06:21:04 2022] GET /identity/v3/projects/df345274c3b840b486c303c6b301052e => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:04.927322 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ef4fbd-bdde-4d71-8fca-3ca8fa4da49b None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:05.061842 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ef4fbd-bdde-4d71-8fca-3ca8fa4da49b None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:05.086379 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:05.113732 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a4111e-6c95-4246-a62f-2065e7826dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:05.114355 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a4111e-6c95-4246-a62f-2065e7826dec None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:05.114673 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a4111e-6c95-4246-a62f-2065e7826dec None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:05.114918 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a4111e-6c95-4246-a62f-2065e7826dec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:05.118101 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a4111e-6c95-4246-a62f-2065e7826dec None admin] RBAC: Adding query filter params (role.id=fcfa05f1d2b1474bb4fc20fc988cacd9, scope.project.id=df345274c3b840b486c303c6b301052e, user.id=f77fd3d9bc3c45bcafb43f716824579a) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:05.118378 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a4111e-6c95-4246-a62f-2065e7826dec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:05.119861 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a4111e-6c95-4246-a62f-2065e7826dec None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:05.126381 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 68/137] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 24 06:21:04 2022] GET /identity/v3/role_assignments?user.id=f77fd3d9bc3c45bcafb43f716824579a&scope.project.id=df345274c3b840b486c303c6b301052e&role.id=fcfa05f1d2b1474bb4fc20fc988cacd9 => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:05.782843 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2752f2d-c823-43d2-8730-a25279df4888 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:05.783898 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2752f2d-c823-43d2-8730-a25279df4888 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:05.784405 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2752f2d-c823-43d2-8730-a25279df4888 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:05.786297 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 70/138] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:21:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:21:05.798926 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8240b299-c4ba-47cc-8880-1327793b999b None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:05.799913 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8240b299-c4ba-47cc-8880-1327793b999b None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:05.800393 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8240b299-c4ba-47cc-8880-1327793b999b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:05.834468 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-8240b299-c4ba-47cc-8880-1327793b999b None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:21:05.836118 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-8240b299-c4ba-47cc-8880-1327793b999b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:21:05.930077 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 69/139] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:21:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:21:06.311153 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d032c0-422c-418b-9522-18c03fd3c5de None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:06.311793 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d032c0-422c-418b-9522-18c03fd3c5de None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:06.312825 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d032c0-422c-418b-9522-18c03fd3c5de None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:06.314602 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 71/140] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:21:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:21:06.320333 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03082391-96ca-47b8-8474-eab535560b5e None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:06.320621 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03082391-96ca-47b8-8474-eab535560b5e None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:06.320903 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03082391-96ca-47b8-8474-eab535560b5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:06.350349 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-03082391-96ca-47b8-8474-eab535560b5e None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:21:06.351571 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-03082391-96ca-47b8-8474-eab535560b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:21:06.400746 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cbdf9d-25e7-4d73-8498-16ceae18c3e3 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:06.401408 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cbdf9d-25e7-4d73-8498-16ceae18c3e3 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:06.401886 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cbdf9d-25e7-4d73-8498-16ceae18c3e3 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:06.403897 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 72/141] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:21:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:21:06.409942 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a520e-fc69-4955-b44b-52ff08eb2513 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:06.410542 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a520e-fc69-4955-b44b-52ff08eb2513 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:06.411013 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a520e-fc69-4955-b44b-52ff08eb2513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:06.446277 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-ab9a520e-fc69-4955-b44b-52ff08eb2513 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:21:06.447854 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-ab9a520e-fc69-4955-b44b-52ff08eb2513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:21:06.474867 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 70/142] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:21:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:21:06.495526 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03082391-96ca-47b8-8474-eab535560b5e None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:06.521551 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:06.590678 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:06.607517 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 73/143] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:21:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 06:21:06.707193 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03082391-96ca-47b8-8474-eab535560b5e None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:06.753829 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00117fd2-f4c1-493b-baac-cc33557da6cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:06.754464 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00117fd2-f4c1-493b-baac-cc33557da6cf None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:06.754750 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00117fd2-f4c1-493b-baac-cc33557da6cf None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:06.755023 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00117fd2-f4c1-493b-baac-cc33557da6cf None admin] PATH_INFO: `/v3/roles/member` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:06.776946 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00117fd2-f4c1-493b-baac-cc33557da6cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:21:06.782969 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-00117fd2-f4c1-493b-baac-cc33557da6cf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 06:21:06.782969 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:21:06.782969 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:21:06.782969 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:21:06.782969 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:21:06.782969 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:21:06.782969 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:21:06.782969 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:21:06.782969 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:21:06.782969 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:21:06.782969 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:21:06.782969 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:21:06.782969 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:21:06.782969 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:21:06.782969 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:21:06.782969 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:21:06.783964 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:21:06.783964 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:21:06.783964 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:21:06.783964 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:21:06.783964 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:21:06.783964 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:21:06.783964 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:21:06.783964 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:21:06.783964 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:21:06.783964 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:21:06.783964 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:21:06.783964 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:21:06.783964 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:21:06.783964 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:21:06.783964 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:21:06.783964 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:21:06.784740 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:21:06.784740 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:21:06.784740 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:21:06.784740 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:21:06.784740 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:21:06.784740 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:21:06.784740 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:21:06.784740 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:21:06.784740 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:21:06.784740 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:21:06.784740 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 06:21:06.784740 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:21:06.787311 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 71/144] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 24 06:21:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:21:06.793133 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9a520e-fc69-4955-b44b-52ff08eb2513 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:06.839994 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00117fd2-f4c1-493b-baac-cc33557da6cf None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:07.031898 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9a520e-fc69-4955-b44b-52ff08eb2513 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:07.050838 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00117fd2-f4c1-493b-baac-cc33557da6cf None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:07.065546 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:07.082647 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7554f02-dfe2-482a-a022-8ee11478688b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:07.083368 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7554f02-dfe2-482a-a022-8ee11478688b None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:07.083652 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7554f02-dfe2-482a-a022-8ee11478688b None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:07.083928 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7554f02-dfe2-482a-a022-8ee11478688b None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:07.084651 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7554f02-dfe2-482a-a022-8ee11478688b None admin] RBAC: Adding query filter params (name=member) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:07.085030 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7554f02-dfe2-482a-a022-8ee11478688b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:07.086549 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7554f02-dfe2-482a-a022-8ee11478688b None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:07.097577 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 74/145] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Aug 24 06:21:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:07.098558 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7554f02-dfe2-482a-a022-8ee11478688b None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:07.109365 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8ad92c-400a-4c60-a945-a86a604e16b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:07.110011 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8ad92c-400a-4c60-a945-a86a604e16b0 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:07.110292 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8ad92c-400a-4c60-a945-a86a604e16b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:07.110564 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8ad92c-400a-4c60-a945-a86a604e16b0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:07.122328 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8ad92c-400a-4c60-a945-a86a604e16b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:21:07.128461 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-7e8ad92c-400a-4c60-a945-a86a604e16b0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 06:21:07.128461 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:21:07.128461 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:21:07.128461 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:21:07.128461 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:21:07.128461 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:21:07.128461 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:21:07.128461 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:21:07.128461 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:21:07.128461 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:21:07.128461 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:21:07.128461 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:21:07.128461 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:21:07.128461 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:21:07.128461 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:21:07.128461 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:21:07.129384 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:21:07.129384 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:21:07.129384 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:21:07.129384 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:21:07.129384 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:21:07.129384 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:21:07.129384 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:21:07.129384 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:21:07.129384 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:21:07.129384 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:21:07.129384 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:21:07.129384 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:21:07.129384 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:21:07.129384 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:21:07.129384 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:21:07.129384 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:21:07.130137 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:21:07.130137 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:21:07.130137 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:21:07.130137 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:21:07.130137 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:21:07.130137 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:21:07.130137 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:21:07.130137 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:21:07.130137 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:21:07.130137 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:21:07.130137 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 06:21:07.130137 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:21:07.130137 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 72/146] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Aug 24 06:21:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:21:07.131260 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98868520-f4a0-40c6-a46a-d6e020a36664 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:07.131421 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98868520-f4a0-40c6-a46a-d6e020a36664 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:07.131696 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98868520-f4a0-40c6-a46a-d6e020a36664 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:07.141216 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:07.141384 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 73/147] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:21:07 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:21:07.141384 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98868520-f4a0-40c6-a46a-d6e020a36664 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:07.235173 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7554f02-dfe2-482a-a022-8ee11478688b None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:07.264085 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98868520-f4a0-40c6-a46a-d6e020a36664 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:07.281044 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1b2b82-9073-4eb9-b89f-bd8952a81b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:07.281748 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1b2b82-9073-4eb9-b89f-bd8952a81b09 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:07.282549 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1b2b82-9073-4eb9-b89f-bd8952a81b09 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:07.282818 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1b2b82-9073-4eb9-b89f-bd8952a81b09 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:07.300447 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1b2b82-9073-4eb9-b89f-bd8952a81b09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:21:07.306114 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-1e1b2b82-9073-4eb9-b89f-bd8952a81b09 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:21:07.306114 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:21:07.306114 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:21:07.306114 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:21:07.306114 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:21:07.306114 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:21:07.306114 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:21:07.306114 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:21:07.306114 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:21:07.306114 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:21:07.306114 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:21:07.306114 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:21:07.306114 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:21:07.306114 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:21:07.306114 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:21:07.306114 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:21:07.306988 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:21:07.306988 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:21:07.306988 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:21:07.306988 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:21:07.306988 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:21:07.306988 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:21:07.306988 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:21:07.306988 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:21:07.306988 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:21:07.306988 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:21:07.306988 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:21:07.306988 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:21:07.306988 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:21:07.306988 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:21:07.306988 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:21:07.306988 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:21:07.307718 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:21:07.307718 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:21:07.307718 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:21:07.307718 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:21:07.307718 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:21:07.307718 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:21:07.307718 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:21:07.307718 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:21:07.307718 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:21:07.307718 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:21:07.307718 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:21:07.307718 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:21:07.307718 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 75/148] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Aug 24 06:21:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:21:07.308367 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1b2b82-9073-4eb9-b89f-bd8952a81b09 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:07.319414 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37237581-80b7-48d8-a47c-7148546500a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:07.320068 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37237581-80b7-48d8-a47c-7148546500a4 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:07.320366 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37237581-80b7-48d8-a47c-7148546500a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:07.320637 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37237581-80b7-48d8-a47c-7148546500a4 None admin] PATH_INFO: `/v3/users/b751994501284709b7f528880544d3b4` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:07.321849 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37237581-80b7-48d8-a47c-7148546500a4 None admin] RBAC: Authorizing `identity:get_user(user_id=b751994501284709b7f528880544d3b4)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:07.323490 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37237581-80b7-48d8-a47c-7148546500a4 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:07.325403 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 74/149] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 24 06:21:07 2022] GET /identity/v3/users/b751994501284709b7f528880544d3b4 => generated 275 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:07.330701 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cb8c47-dca3-4cb7-bd29-a7637a9b995f None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:07.330701 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cb8c47-dca3-4cb7-bd29-a7637a9b995f None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:07.330701 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cb8c47-dca3-4cb7-bd29-a7637a9b995f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:07.348604 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-36cb8c47-dca3-4cb7-bd29-a7637a9b995f None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:21:07.350162 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-36cb8c47-dca3-4cb7-bd29-a7637a9b995f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:21:07.372806 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:07.425159 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 75/150] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:21:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:21:07.431567 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cb8c47-dca3-4cb7-bd29-a7637a9b995f None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:07.432145 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:07.463169 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1b2b82-9073-4eb9-b89f-bd8952a81b09 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:07.507963 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6547cf-cdb8-4516-bce6-a9edddb345e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:07.508658 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6547cf-cdb8-4516-bce6-a9edddb345e1 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:07.508951 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6547cf-cdb8-4516-bce6-a9edddb345e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:07.509251 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6547cf-cdb8-4516-bce6-a9edddb345e1 None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:07.509935 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6547cf-cdb8-4516-bce6-a9edddb345e1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:07.510240 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6547cf-cdb8-4516-bce6-a9edddb345e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:07.511745 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6547cf-cdb8-4516-bce6-a9edddb345e1 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:07.522246 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 76/151] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 24 06:21:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:07.523948 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6547cf-cdb8-4516-bce6-a9edddb345e1 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:07.562080 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cb8c47-dca3-4cb7-bd29-a7637a9b995f None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:07.625988 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c50ba2-f064-47d2-bb4a-ccd654c6c812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:07.625988 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c50ba2-f064-47d2-bb4a-ccd654c6c812 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:07.626413 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c50ba2-f064-47d2-bb4a-ccd654c6c812 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:07.626532 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c50ba2-f064-47d2-bb4a-ccd654c6c812 None admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:07.627158 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c50ba2-f064-47d2-bb4a-ccd654c6c812 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:07.627459 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c50ba2-f064-47d2-bb4a-ccd654c6c812 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:07.628870 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c50ba2-f064-47d2-bb4a-ccd654c6c812 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:07.644799 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 76/152] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 24 06:21:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:07.645967 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3437c6de-931c-4209-8dba-2d569418d728 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:07.646264 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3437c6de-931c-4209-8dba-2d569418d728 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:07.646538 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3437c6de-931c-4209-8dba-2d569418d728 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:07.651858 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 77/153] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:21:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:21:07.651858 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3437c6de-931c-4209-8dba-2d569418d728 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:07.672436 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:07.697963 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6547cf-cdb8-4516-bce6-a9edddb345e1 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:07.744408 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6adb4b-a00d-46f1-a022-05c491ebb227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:07.745494 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6adb4b-a00d-46f1-a022-05c491ebb227 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:07.745972 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6adb4b-a00d-46f1-a022-05c491ebb227 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:07.746420 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6adb4b-a00d-46f1-a022-05c491ebb227 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:07.765713 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:07.775437 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3437c6de-931c-4209-8dba-2d569418d728 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:07.814787 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef6ac47-6750-4501-9d26-aa0a96f3901f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:07.815444 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef6ac47-6750-4501-9d26-aa0a96f3901f None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:07.815733 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef6ac47-6750-4501-9d26-aa0a96f3901f None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:07.816104 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef6ac47-6750-4501-9d26-aa0a96f3901f None admin] PATH_INFO: `/v3/users/b751994501284709b7f528880544d3b4` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:07.817760 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef6ac47-6750-4501-9d26-aa0a96f3901f None admin] RBAC: Authorizing `identity:get_user(user_id=b751994501284709b7f528880544d3b4)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:07.819258 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef6ac47-6750-4501-9d26-aa0a96f3901f None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:07.821001 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 78/154] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 24 06:21:07 2022] GET /identity/v3/users/b751994501284709b7f528880544d3b4 => generated 275 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:07.823734 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6adb4b-a00d-46f1-a022-05c491ebb227 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:07.825420 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6adb4b-a00d-46f1-a022-05c491ebb227 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:07.826324 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef6ac47-6750-4501-9d26-aa0a96f3901f None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:07.931557 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:07.940385 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 77/155] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 24 06:21:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 06:21:07.943812 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405036b2-50ea-4d0a-98a2-42b5edef886c None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:07.943812 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405036b2-50ea-4d0a-98a2-42b5edef886c None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:07.944306 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405036b2-50ea-4d0a-98a2-42b5edef886c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:07.948400 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef6ac47-6750-4501-9d26-aa0a96f3901f None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:07.963117 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-405036b2-50ea-4d0a-98a2-42b5edef886c None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:21:07.964370 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-405036b2-50ea-4d0a-98a2-42b5edef886c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:21:07.993203 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8969a41-a323-4727-b8c4-2a051b0186ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:07.993865 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8969a41-a323-4727-b8c4-2a051b0186ff None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:07.994148 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8969a41-a323-4727-b8c4-2a051b0186ff None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:07.994427 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8969a41-a323-4727-b8c4-2a051b0186ff None admin] PATH_INFO: `/v3/users/029600e45e6b473db6751098bdba45e2` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:07.995607 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8969a41-a323-4727-b8c4-2a051b0186ff None admin] RBAC: Authorizing `identity:get_user(user_id=029600e45e6b473db6751098bdba45e2)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:07.997072 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8969a41-a323-4727-b8c4-2a051b0186ff None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:07.998882 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 79/156] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 24 06:21:07 2022] GET /identity/v3/users/029600e45e6b473db6751098bdba45e2 => generated 247 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:08.002092 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8969a41-a323-4727-b8c4-2a051b0186ff None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:08.021403 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:08.047858 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 78/157] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:21:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:21:08.049059 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405036b2-50ea-4d0a-98a2-42b5edef886c None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:08.132523 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8969a41-a323-4727-b8c4-2a051b0186ff None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:08.172524 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b1109a-a525-4cf5-9ecd-e08c9702111e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:08.173154 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b1109a-a525-4cf5-9ecd-e08c9702111e None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:08.173349 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405036b2-50ea-4d0a-98a2-42b5edef886c None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:08.173488 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b1109a-a525-4cf5-9ecd-e08c9702111e None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:08.173783 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b1109a-a525-4cf5-9ecd-e08c9702111e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:08.187325 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b1109a-a525-4cf5-9ecd-e08c9702111e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:21:08.192942 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-e4b1109a-a525-4cf5-9ecd-e08c9702111e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 06:21:08.192942 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:21:08.192942 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:21:08.192942 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:21:08.192942 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:21:08.192942 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:21:08.192942 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:21:08.192942 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:21:08.192942 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:21:08.192942 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:21:08.192942 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:21:08.192942 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:21:08.192942 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:21:08.192942 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:21:08.192942 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:21:08.192942 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:21:08.193972 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:21:08.193972 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:21:08.193972 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:21:08.193972 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:21:08.193972 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:21:08.193972 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:21:08.193972 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:21:08.193972 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:21:08.193972 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:21:08.193972 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:21:08.193972 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:21:08.193972 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:21:08.193972 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:21:08.193972 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:21:08.193972 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:21:08.193972 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:21:08.194757 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:21:08.194757 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:21:08.194757 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:21:08.194757 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:21:08.194757 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:21:08.194757 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:21:08.194757 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:21:08.194757 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:21:08.194757 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:21:08.194757 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:21:08.194757 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 06:21:08.194757 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:21:08.194757 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 80/158] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 24 06:21:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:21:08.199648 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b1109a-a525-4cf5-9ecd-e08c9702111e None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:08.217764 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:08.218834 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0304f7e5-a878-4295-9a07-f078d8a36929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:08.220123 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0304f7e5-a878-4295-9a07-f078d8a36929 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:08.220819 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0304f7e5-a878-4295-9a07-f078d8a36929 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:08.221204 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0304f7e5-a878-4295-9a07-f078d8a36929 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:08.273749 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:08.298649 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0304f7e5-a878-4295-9a07-f078d8a36929 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:08.300140 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0304f7e5-a878-4295-9a07-f078d8a36929 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:08.320842 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b1109a-a525-4cf5-9ecd-e08c9702111e None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:08.362390 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c9bc04-6d13-4ae0-a2df-1e9df7f8143a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:08.363024 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c9bc04-6d13-4ae0-a2df-1e9df7f8143a None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:08.363310 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c9bc04-6d13-4ae0-a2df-1e9df7f8143a None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:08.363578 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c9bc04-6d13-4ae0-a2df-1e9df7f8143a None admin] PATH_INFO: `/v3/projects/df345274c3b840b486c303c6b301052e` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:08.364709 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c9bc04-6d13-4ae0-a2df-1e9df7f8143a None admin] RBAC: Authorizing `identity:get_project(project_id=df345274c3b840b486c303c6b301052e)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:08.366219 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c9bc04-6d13-4ae0-a2df-1e9df7f8143a None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:08.368198 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 81/159] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 24 06:21:08 2022] GET /identity/v3/projects/df345274c3b840b486c303c6b301052e => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:08.368672 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c9bc04-6d13-4ae0-a2df-1e9df7f8143a None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:08.408441 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 79/160] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 24 06:21:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 06:21:08.409617 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1178efba-c841-47b5-962a-8c27c62088d5 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:08.409898 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1178efba-c841-47b5-962a-8c27c62088d5 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:08.410165 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1178efba-c841-47b5-962a-8c27c62088d5 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:08.411433 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 80/161] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:21:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:21:08.416492 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1178efba-c841-47b5-962a-8c27c62088d5 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:08.457004 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:08.475439 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c9bc04-6d13-4ae0-a2df-1e9df7f8143a None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:08.504829 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:08.511152 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988d426e-870a-4b8b-9dfb-f188b5a16718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:08.512607 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988d426e-870a-4b8b-9dfb-f188b5a16718 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:08.512607 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988d426e-870a-4b8b-9dfb-f188b5a16718 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:08.512607 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988d426e-870a-4b8b-9dfb-f188b5a16718 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:08.526335 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1178efba-c841-47b5-962a-8c27c62088d5 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:08.564202 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8972225b-3409-46ad-b0ff-efe65c7d8bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:08.564814 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8972225b-3409-46ad-b0ff-efe65c7d8bd9 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:08.565094 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8972225b-3409-46ad-b0ff-efe65c7d8bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:08.565405 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8972225b-3409-46ad-b0ff-efe65c7d8bd9 None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:08.566078 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8972225b-3409-46ad-b0ff-efe65c7d8bd9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:08.566387 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8972225b-3409-46ad-b0ff-efe65c7d8bd9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:08.567760 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8972225b-3409-46ad-b0ff-efe65c7d8bd9 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:08.576206 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 81/162] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Aug 24 06:21:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:08.577052 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8972225b-3409-46ad-b0ff-efe65c7d8bd9 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:08.588678 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988d426e-870a-4b8b-9dfb-f188b5a16718 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:08.590176 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988d426e-870a-4b8b-9dfb-f188b5a16718 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:08.677381 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:08.684854 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8972225b-3409-46ad-b0ff-efe65c7d8bd9 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:08.698429 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 82/163] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 24 06:21:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 06:21:08.703372 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988d426e-870a-4b8b-9dfb-f188b5a16718 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:08.723592 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8799652-1f9a-4343-9055-e52894c271b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:08.724238 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8799652-1f9a-4343-9055-e52894c271b3 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:08.724520 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8799652-1f9a-4343-9055-e52894c271b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:08.724817 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8799652-1f9a-4343-9055-e52894c271b3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:08.726008 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8799652-1f9a-4343-9055-e52894c271b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:08.727459 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8799652-1f9a-4343-9055-e52894c271b3 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:08.729039 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 82/164] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:21:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:08.729844 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8799652-1f9a-4343-9055-e52894c271b3 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:08.751844 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:08.818781 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988d426e-870a-4b8b-9dfb-f188b5a16718 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:08.842094 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8799652-1f9a-4343-9055-e52894c271b3 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:08.854212 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82145654-7f04-46df-9a4c-a4705f5c787d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:08.854824 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82145654-7f04-46df-9a4c-a4705f5c787d None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:08.855103 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82145654-7f04-46df-9a4c-a4705f5c787d None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:08.855374 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82145654-7f04-46df-9a4c-a4705f5c787d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:08.856114 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82145654-7f04-46df-9a4c-a4705f5c787d None admin] RBAC: Adding query filter params (role.id=e1f9b167c51c4a388cbc9594db8a465d, scope.project.id=df345274c3b840b486c303c6b301052e, user.id=b751994501284709b7f528880544d3b4) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:08.856410 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82145654-7f04-46df-9a4c-a4705f5c787d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:08.857894 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82145654-7f04-46df-9a4c-a4705f5c787d None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:08.864461 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 83/165] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 24 06:21:08 2022] GET /identity/v3/role_assignments?user.id=b751994501284709b7f528880544d3b4&scope.project.id=df345274c3b840b486c303c6b301052e&role.id=e1f9b167c51c4a388cbc9594db8a465d => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:08.865710 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2d4547-b032-4356-a585-0619c45daa76 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:08.865997 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2d4547-b032-4356-a585-0619c45daa76 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:08.866271 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2d4547-b032-4356-a585-0619c45daa76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:08.888241 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8469a5e-aeb8-4304-89b4-fb630a11d0ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:08.889254 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8469a5e-aeb8-4304-89b4-fb630a11d0ca None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:08.889691 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8469a5e-aeb8-4304-89b4-fb630a11d0ca None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:08.890642 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8469a5e-aeb8-4304-89b4-fb630a11d0ca None admin] PATH_INFO: `/v3/projects/df345274c3b840b486c303c6b301052e` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:08.892514 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8469a5e-aeb8-4304-89b4-fb630a11d0ca None admin] RBAC: Authorizing `identity:get_project(project_id=df345274c3b840b486c303c6b301052e)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:08.893100 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-0a2d4547-b032-4356-a585-0619c45daa76 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:21:08.894357 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-0a2d4547-b032-4356-a585-0619c45daa76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:21:08.894978 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8469a5e-aeb8-4304-89b4-fb630a11d0ca None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:08.898305 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 83/166] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 24 06:21:08 2022] GET /identity/v3/projects/df345274c3b840b486c303c6b301052e => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:08.899143 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8469a5e-aeb8-4304-89b4-fb630a11d0ca None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:08.950611 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:08.981744 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 84/167] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:21:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:21:08.982625 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2d4547-b032-4356-a585-0619c45daa76 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:09.045565 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:09.059953 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8469a5e-aeb8-4304-89b4-fb630a11d0ca None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:09.107414 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e623fe89-e863-4802-9d2f-83495660165b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:09.108204 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e623fe89-e863-4802-9d2f-83495660165b None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:09.108492 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e623fe89-e863-4802-9d2f-83495660165b None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:09.109505 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e623fe89-e863-4802-9d2f-83495660165b None admin] PATH_INFO: `/v3/users/b751994501284709b7f528880544d3b4` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:09.110342 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e623fe89-e863-4802-9d2f-83495660165b None admin] RBAC: Authorizing `identity:get_user(user_id=b751994501284709b7f528880544d3b4)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:09.111780 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e623fe89-e863-4802-9d2f-83495660165b None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:09.112182 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2d4547-b032-4356-a585-0619c45daa76 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:09.114385 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 84/168] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 24 06:21:08 2022] GET /identity/v3/users/b751994501284709b7f528880544d3b4 => generated 275 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:09.114385 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e623fe89-e863-4802-9d2f-83495660165b None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:09.160530 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aa7323-e05c-4f3f-a78f-9e0ef901ce34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:09.161267 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aa7323-e05c-4f3f-a78f-9e0ef901ce34 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:09.161652 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aa7323-e05c-4f3f-a78f-9e0ef901ce34 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:09.161830 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aa7323-e05c-4f3f-a78f-9e0ef901ce34 None admin] PATH_INFO: `/v3/projects/df345274c3b840b486c303c6b301052e` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:09.163549 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aa7323-e05c-4f3f-a78f-9e0ef901ce34 None admin] RBAC: Authorizing `identity:get_project(project_id=df345274c3b840b486c303c6b301052e)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:09.164463 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aa7323-e05c-4f3f-a78f-9e0ef901ce34 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:09.166312 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 85/169] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 24 06:21:08 2022] GET /identity/v3/projects/df345274c3b840b486c303c6b301052e => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:09.166921 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aa7323-e05c-4f3f-a78f-9e0ef901ce34 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:09.219770 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:09.240602 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e623fe89-e863-4802-9d2f-83495660165b None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:09.279853 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b51ee1-2795-41ee-819a-70f0051e5906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:09.280478 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b51ee1-2795-41ee-819a-70f0051e5906 None admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:09.281270 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b51ee1-2795-41ee-819a-70f0051e5906 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:09.281609 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b51ee1-2795-41ee-819a-70f0051e5906 None admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:09.282271 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b51ee1-2795-41ee-819a-70f0051e5906 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:09.283716 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b51ee1-2795-41ee-819a-70f0051e5906 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:09.286830 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aa7323-e05c-4f3f-a78f-9e0ef901ce34 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:09.321802 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4af8d5-0794-4bd3-94c2-3200f358b6a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:09.322406 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4af8d5-0794-4bd3-94c2-3200f358b6a2 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:09.322688 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4af8d5-0794-4bd3-94c2-3200f358b6a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:09.322961 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4af8d5-0794-4bd3-94c2-3200f358b6a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:09.323771 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4af8d5-0794-4bd3-94c2-3200f358b6a2 None admin] RBAC: Adding query filter params (role.id=b71c87e12dc1452ea3c99a7bf28585ca, scope.project.id=df345274c3b840b486c303c6b301052e, user.id=b751994501284709b7f528880544d3b4) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:09.324080 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4af8d5-0794-4bd3-94c2-3200f358b6a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:09.325589 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4af8d5-0794-4bd3-94c2-3200f358b6a2 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:09.332648 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 86/170] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 24 06:21:09 2022] GET /identity/v3/role_assignments?user.id=b751994501284709b7f528880544d3b4&scope.project.id=df345274c3b840b486c303c6b301052e&role.id=b71c87e12dc1452ea3c99a7bf28585ca => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:09.334926 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4af8d5-0794-4bd3-94c2-3200f358b6a2 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:09.389309 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 85/171] 10.222.0.47 () {60 vars in 1132 bytes} [Wed Aug 24 06:21:09 2022] POST /identity/v3/users => generated 283 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:21:09.390836 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b51ee1-2795-41ee-819a-70f0051e5906 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:09.393878 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:09.481652 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4af8d5-0794-4bd3-94c2-3200f358b6a2 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:09.493891 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:09.539175 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaf31d3-5b64-4a8a-91a2-1badf45e3769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:09.539936 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaf31d3-5b64-4a8a-91a2-1badf45e3769 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:09.541100 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaf31d3-5b64-4a8a-91a2-1badf45e3769 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:09.541100 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaf31d3-5b64-4a8a-91a2-1badf45e3769 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:09.548453 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b51ee1-2795-41ee-819a-70f0051e5906 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:09.594045 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86afa282-d20b-481f-836f-49121cc1a20c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:09.594678 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86afa282-d20b-481f-836f-49121cc1a20c None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:09.595014 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86afa282-d20b-481f-836f-49121cc1a20c None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:09.595238 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86afa282-d20b-481f-836f-49121cc1a20c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:09.595988 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86afa282-d20b-481f-836f-49121cc1a20c None admin] RBAC: Adding query filter params (role.id=486144ad1bcd417e86acdbc1d14327bf, scope.project.id=df345274c3b840b486c303c6b301052e, user.id=029600e45e6b473db6751098bdba45e2) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:09.596304 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86afa282-d20b-481f-836f-49121cc1a20c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:09.598333 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86afa282-d20b-481f-836f-49121cc1a20c None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:09.607496 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 86/172] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 24 06:21:09 2022] GET /identity/v3/role_assignments?user.id=029600e45e6b473db6751098bdba45e2&scope.project.id=df345274c3b840b486c303c6b301052e&role.id=486144ad1bcd417e86acdbc1d14327bf => generated 608 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:09.608358 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86afa282-d20b-481f-836f-49121cc1a20c None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:09.649515 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaf31d3-5b64-4a8a-91a2-1badf45e3769 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:09.651418 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaf31d3-5b64-4a8a-91a2-1badf45e3769 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:09.712221 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:09.780254 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86afa282-d20b-481f-836f-49121cc1a20c None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:09.807315 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 87/173] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 24 06:21:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 06:21:09.817724 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaf31d3-5b64-4a8a-91a2-1badf45e3769 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:09.831987 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:09.846847 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19940ff6-72a4-47fd-9b99-ab295b7222f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:09.847520 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19940ff6-72a4-47fd-9b99-ab295b7222f9 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:09.847798 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19940ff6-72a4-47fd-9b99-ab295b7222f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:09.848057 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19940ff6-72a4-47fd-9b99-ab295b7222f9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:09.863570 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19940ff6-72a4-47fd-9b99-ab295b7222f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:21:09.869169 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-19940ff6-72a4-47fd-9b99-ab295b7222f9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 06:21:09.869169 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:21:09.869169 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:21:09.869169 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:21:09.869169 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:21:09.869169 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:21:09.869169 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:21:09.869169 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:21:09.869169 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:21:09.869169 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:21:09.869169 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:21:09.869169 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:21:09.869169 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:21:09.869169 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:21:09.869169 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:21:09.869169 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:21:09.870117 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:21:09.870117 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:21:09.870117 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:21:09.870117 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:21:09.870117 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:21:09.870117 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:21:09.870117 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:21:09.870117 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:21:09.870117 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:21:09.870117 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:21:09.870117 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:21:09.870117 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:21:09.870117 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:21:09.870117 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:21:09.870117 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:21:09.870117 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:21:09.870900 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:21:09.870900 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:21:09.870900 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:21:09.870900 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:21:09.870900 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:21:09.870900 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:21:09.870900 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:21:09.870900 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:21:09.870900 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:21:09.870900 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:21:09.870900 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 06:21:09.870900 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:21:09.876282 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 87/174] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 24 06:21:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:21:09.877675 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19940ff6-72a4-47fd-9b99-ab295b7222f9 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:09.965713 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaf31d3-5b64-4a8a-91a2-1badf45e3769 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:10.009130 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19940ff6-72a4-47fd-9b99-ab295b7222f9 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:10.018629 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48797629-bb73-4b2c-ad58-8e519ed32753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:10.019329 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48797629-bb73-4b2c-ad58-8e519ed32753 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:10.020263 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48797629-bb73-4b2c-ad58-8e519ed32753 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:10.020552 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48797629-bb73-4b2c-ad58-8e519ed32753 None admin] PATH_INFO: `/v3/projects/58c030ee4f10414aa5f379a1c9cab0be` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:10.021806 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48797629-bb73-4b2c-ad58-8e519ed32753 None admin] RBAC: Authorizing `identity:get_project(project_id=58c030ee4f10414aa5f379a1c9cab0be)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:10.023785 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48797629-bb73-4b2c-ad58-8e519ed32753 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:10.026368 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 88/175] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 24 06:21:09 2022] GET /identity/v3/projects/58c030ee4f10414aa5f379a1c9cab0be => generated 312 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:21:10.033356 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48797629-bb73-4b2c-ad58-8e519ed32753 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:10.051795 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:10.065211 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554b4fd0-8734-4cdd-b800-37616aa9c009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:10.065810 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554b4fd0-8734-4cdd-b800-37616aa9c009 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:10.066092 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554b4fd0-8734-4cdd-b800-37616aa9c009 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:10.069154 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554b4fd0-8734-4cdd-b800-37616aa9c009 None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:10.069154 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554b4fd0-8734-4cdd-b800-37616aa9c009 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:10.069154 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554b4fd0-8734-4cdd-b800-37616aa9c009 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:10.069154 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554b4fd0-8734-4cdd-b800-37616aa9c009 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:10.080563 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 88/176] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Aug 24 06:21:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:21:10.086010 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554b4fd0-8734-4cdd-b800-37616aa9c009 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:10.153557 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:10.186390 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48797629-bb73-4b2c-ad58-8e519ed32753 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:10.231364 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554b4fd0-8734-4cdd-b800-37616aa9c009 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:10.247638 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b49f0c-b487-435a-93f0-5628d999a579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:10.248533 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b49f0c-b487-435a-93f0-5628d999a579 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:10.248963 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b49f0c-b487-435a-93f0-5628d999a579 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:10.249920 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b49f0c-b487-435a-93f0-5628d999a579 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:10.253426 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b49f0c-b487-435a-93f0-5628d999a579 None admin] RBAC: Adding query filter params (role.id=e1f9b167c51c4a388cbc9594db8a465d, scope.project.id=58c030ee4f10414aa5f379a1c9cab0be, user.id=b751994501284709b7f528880544d3b4) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:10.253913 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b49f0c-b487-435a-93f0-5628d999a579 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:10.255543 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b49f0c-b487-435a-93f0-5628d999a579 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:10.270522 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 89/177] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 24 06:21:10 2022] GET /identity/v3/role_assignments?user.id=b751994501284709b7f528880544d3b4&scope.project.id=58c030ee4f10414aa5f379a1c9cab0be&role.id=e1f9b167c51c4a388cbc9594db8a465d => generated 608 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:10.303987 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8b0955-e032-4a25-910b-263cc7eabb12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:10.304901 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b0955-e032-4a25-910b-263cc7eabb12 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:10.305350 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b0955-e032-4a25-910b-263cc7eabb12 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:10.305780 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b0955-e032-4a25-910b-263cc7eabb12 None admin] PATH_INFO: `/v3/users/f77fd3d9bc3c45bcafb43f716824579a` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:10.307158 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8b0955-e032-4a25-910b-263cc7eabb12 None admin] RBAC: Authorizing `identity:get_user(user_id=f77fd3d9bc3c45bcafb43f716824579a)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:10.308784 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8b0955-e032-4a25-910b-263cc7eabb12 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:10.311034 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 89/178] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 24 06:21:10 2022] GET /identity/v3/users/f77fd3d9bc3c45bcafb43f716824579a => generated 289 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:21:10.318009 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b49f0c-b487-435a-93f0-5628d999a579 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:10.465718 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b49f0c-b487-435a-93f0-5628d999a579 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:10.508348 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9278b6a8-60b8-4212-8e9f-8dce9c95e0d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:10.509288 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9278b6a8-60b8-4212-8e9f-8dce9c95e0d9 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:10.509708 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9278b6a8-60b8-4212-8e9f-8dce9c95e0d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:10.510121 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9278b6a8-60b8-4212-8e9f-8dce9c95e0d9 None admin] PATH_INFO: `/v3/projects/df345274c3b840b486c303c6b301052e` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:10.511448 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9278b6a8-60b8-4212-8e9f-8dce9c95e0d9 None admin] RBAC: Authorizing `identity:get_project(project_id=df345274c3b840b486c303c6b301052e)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:10.513054 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9278b6a8-60b8-4212-8e9f-8dce9c95e0d9 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:10.515167 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 90/179] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 24 06:21:10 2022] GET /identity/v3/projects/df345274c3b840b486c303c6b301052e => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:10.523393 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8b0955-e032-4a25-910b-263cc7eabb12 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:10.600453 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:10.654126 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8b0955-e032-4a25-910b-263cc7eabb12 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:10.705080 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2def8f0-08f4-493f-8687-1e1b451f5c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:10.705963 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2def8f0-08f4-493f-8687-1e1b451f5c9d None admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:10.706343 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2def8f0-08f4-493f-8687-1e1b451f5c9d None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:10.706736 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2def8f0-08f4-493f-8687-1e1b451f5c9d None admin] PATH_INFO: `/v3/projects/df345274c3b840b486c303c6b301052e/users/f77fd3d9bc3c45bcafb43f716824579a/roles/fcfa05f1d2b1474bb4fc20fc988cacd9` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:10.708814 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2def8f0-08f4-493f-8687-1e1b451f5c9d None admin] RBAC: Authorizing `identity:create_grant(project_id=df345274c3b840b486c303c6b301052e, user_id=f77fd3d9bc3c45bcafb43f716824579a, role_id=fcfa05f1d2b1474bb4fc20fc988cacd9)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:10.710399 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2def8f0-08f4-493f-8687-1e1b451f5c9d None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:10.781155 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 90/180] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Aug 24 06:21:10 2022] PUT /identity/v3/projects/df345274c3b840b486c303c6b301052e/users/f77fd3d9bc3c45bcafb43f716824579a/roles/fcfa05f1d2b1474bb4fc20fc988cacd9 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:21:11.964712 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f863301-039f-460e-9cd7-be4e4d15414f None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:11.965228 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f863301-039f-460e-9cd7-be4e4d15414f None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:11.965228 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f863301-039f-460e-9cd7-be4e4d15414f None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:11.966711 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 91/181] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:21:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:21:11.977388 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2deb9f-2d14-4cce-8851-2b2cebf22d59 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:11.977388 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2deb9f-2d14-4cce-8851-2b2cebf22d59 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:11.977388 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2deb9f-2d14-4cce-8851-2b2cebf22d59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:11.979507 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ee5f4-9d65-42e0-ba5c-d00b5d040976 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:11.979789 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ee5f4-9d65-42e0-ba5c-d00b5d040976 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:11.980065 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ee5f4-9d65-42e0-ba5c-d00b5d040976 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:11.981703 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 92/182] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:21:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:21:11.990713 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc6e58e-e1f1-4b95-aed8-80b94ee8c426 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:11.991145 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc6e58e-e1f1-4b95-aed8-80b94ee8c426 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:11.991519 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc6e58e-e1f1-4b95-aed8-80b94ee8c426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:12.000467 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-9b2deb9f-2d14-4cce-8851-2b2cebf22d59 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:21:12.001962 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-9b2deb9f-2d14-4cce-8851-2b2cebf22d59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:21:12.011383 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-cbc6e58e-e1f1-4b95-aed8-80b94ee8c426 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:21:12.012777 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-cbc6e58e-e1f1-4b95-aed8-80b94ee8c426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:21:12.015133 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:12.085076 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 91/183] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:21:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:21:12.086289 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 93/184] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:21:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 06:21:12.089454 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3eac34-f542-4aa0-a3d1-cfa746e55b3e None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:12.089943 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3eac34-f542-4aa0-a3d1-cfa746e55b3e None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:12.090353 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3eac34-f542-4aa0-a3d1-cfa746e55b3e None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:12.091853 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 92/185] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:21:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:21:12.103601 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4256b7ec-0839-4911-8e83-b19c1eb77604 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:12.104093 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4256b7ec-0839-4911-8e83-b19c1eb77604 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:12.104540 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4256b7ec-0839-4911-8e83-b19c1eb77604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:12.124082 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-4256b7ec-0839-4911-8e83-b19c1eb77604 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:21:12.125831 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-4256b7ec-0839-4911-8e83-b19c1eb77604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:21:12.212167 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 94/186] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:21:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:21:12.368569 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3eac34-f542-4aa0-a3d1-cfa746e55b3e None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:12.376143 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4256b7ec-0839-4911-8e83-b19c1eb77604 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:12.424140 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:12.461618 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:12.512522 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4256b7ec-0839-4911-8e83-b19c1eb77604 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:12.513048 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3eac34-f542-4aa0-a3d1-cfa746e55b3e None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:12.567296 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b80944f-36d0-46a0-b3c4-23ef2b3a8e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:12.567955 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b80944f-36d0-46a0-b3c4-23ef2b3a8e21 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:12.568105 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0570d845-866c-416b-accd-0d79f24a3d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:12.568218 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b80944f-36d0-46a0-b3c4-23ef2b3a8e21 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:12.568481 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b80944f-36d0-46a0-b3c4-23ef2b3a8e21 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:12.568668 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0570d845-866c-416b-accd-0d79f24a3d9b None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:12.568918 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0570d845-866c-416b-accd-0d79f24a3d9b None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:12.569197 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0570d845-866c-416b-accd-0d79f24a3d9b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:12.580661 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0570d845-866c-416b-accd-0d79f24a3d9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:21:12.582156 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b80944f-36d0-46a0-b3c4-23ef2b3a8e21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:21:12.585956 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-0570d845-866c-416b-accd-0d79f24a3d9b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 06:21:12.585956 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:21:12.585956 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:21:12.585956 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:21:12.585956 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:21:12.585956 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:21:12.585956 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:21:12.585956 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:21:12.585956 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:21:12.585956 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:21:12.585956 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:21:12.585956 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:21:12.585956 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:21:12.585956 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:21:12.585956 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:21:12.585956 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:21:12.586830 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:21:12.586830 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:21:12.586830 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:21:12.586830 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:21:12.586830 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:21:12.586830 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:21:12.586830 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:21:12.586830 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:21:12.586830 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:21:12.586830 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:21:12.586830 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:21:12.586830 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:21:12.586830 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:21:12.586830 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:21:12.586830 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:21:12.586830 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:21:12.587651 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:21:12.587651 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:21:12.587651 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:21:12.587651 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:21:12.587651 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:21:12.587651 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:21:12.587651 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:21:12.587651 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:21:12.587651 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:21:12.587651 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:21:12.587651 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 06:21:12.587651 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:21:12.587651 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 93/187] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Aug 24 06:21:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:21:12.588468 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-9b80944f-36d0-46a0-b3c4-23ef2b3a8e21 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:21:12.588468 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:21:12.588468 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:21:12.588468 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:21:12.588468 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:21:12.588468 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:21:12.588468 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:21:12.588468 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:21:12.588468 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:21:12.588468 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:21:12.588468 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:21:12.588468 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:21:12.588468 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:21:12.588468 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:21:12.588468 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:21:12.588468 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:21:12.589434 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:21:12.589434 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:21:12.589434 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:21:12.589434 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:21:12.589434 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:21:12.589434 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:21:12.589434 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:21:12.589434 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:21:12.589434 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:21:12.589434 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:21:12.589434 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:21:12.589434 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:21:12.589434 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:21:12.589434 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:21:12.589434 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:21:12.589434 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:21:12.590231 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:21:12.590231 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:21:12.590231 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:21:12.590231 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:21:12.590231 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:21:12.590231 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:21:12.590231 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:21:12.590231 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:21:12.590231 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:21:12.590231 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:21:12.590231 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:21:12.590231 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:21:12.590818 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0570d845-866c-416b-accd-0d79f24a3d9b None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:12.590898 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 95/188] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Aug 24 06:21:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:21:12.597483 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b80944f-36d0-46a0-b3c4-23ef2b3a8e21 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:12.715250 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0570d845-866c-416b-accd-0d79f24a3d9b None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:12.722751 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b80944f-36d0-46a0-b3c4-23ef2b3a8e21 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:12.726099 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:12.755254 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:12.760487 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f55c003-68a6-468d-8e76-7053a9532c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:12.760487 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f55c003-68a6-468d-8e76-7053a9532c8f None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:12.760487 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f55c003-68a6-468d-8e76-7053a9532c8f None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:12.760487 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f55c003-68a6-468d-8e76-7053a9532c8f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:12.765255 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f55c003-68a6-468d-8e76-7053a9532c8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:12.765255 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f55c003-68a6-468d-8e76-7053a9532c8f None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:12.765255 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 94/189] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:21:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:12.765255 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f55c003-68a6-468d-8e76-7053a9532c8f None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:12.769695 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c831f040-355f-4766-90d0-3eabb320c575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:12.770303 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c831f040-355f-4766-90d0-3eabb320c575 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:12.770593 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c831f040-355f-4766-90d0-3eabb320c575 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:12.770869 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c831f040-355f-4766-90d0-3eabb320c575 None admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:12.772970 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c831f040-355f-4766-90d0-3eabb320c575 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:12.772970 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c831f040-355f-4766-90d0-3eabb320c575 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:12.773278 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c831f040-355f-4766-90d0-3eabb320c575 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:12.789135 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 96/190] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 24 06:21:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:21:12.789135 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c831f040-355f-4766-90d0-3eabb320c575 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:12.887471 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f55c003-68a6-468d-8e76-7053a9532c8f None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:12.895394 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c831f040-355f-4766-90d0-3eabb320c575 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:12.924207 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40e73b9-7720-431f-9371-0970c30a80bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:12.924960 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40e73b9-7720-431f-9371-0970c30a80bd None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:12.925266 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40e73b9-7720-431f-9371-0970c30a80bd None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:12.925545 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40e73b9-7720-431f-9371-0970c30a80bd None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:12.926223 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40e73b9-7720-431f-9371-0970c30a80bd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:12.926531 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40e73b9-7720-431f-9371-0970c30a80bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:12.927899 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40e73b9-7720-431f-9371-0970c30a80bd None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:12.931778 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abb7abb-e48d-4fe0-8b54-6894c9943410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:12.932393 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abb7abb-e48d-4fe0-8b54-6894c9943410 None admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:12.932674 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abb7abb-e48d-4fe0-8b54-6894c9943410 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:12.932945 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abb7abb-e48d-4fe0-8b54-6894c9943410 None admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:12.933634 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abb7abb-e48d-4fe0-8b54-6894c9943410 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:12.935079 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abb7abb-e48d-4fe0-8b54-6894c9943410 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:12.938563 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 95/191] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 24 06:21:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:12.940585 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40e73b9-7720-431f-9371-0970c30a80bd None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:12.984243 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:13.039969 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 97/192] 10.222.0.47 () {60 vars in 1132 bytes} [Wed Aug 24 06:21:12 2022] POST /identity/v3/users => generated 297 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:21:13.042263 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8facdd1a-96da-45be-84b7-b7dc9cfa363d None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:13.042500 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8facdd1a-96da-45be-84b7-b7dc9cfa363d None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:13.042780 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8facdd1a-96da-45be-84b7-b7dc9cfa363d None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:13.044159 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 98/193] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:21:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:21:13.049079 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8facdd1a-96da-45be-84b7-b7dc9cfa363d None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:13.064708 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40e73b9-7720-431f-9371-0970c30a80bd None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:13.108412 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:13.115411 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e770681-34c0-45c2-b28d-719fd7117043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:13.118850 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e770681-34c0-45c2-b28d-719fd7117043 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:13.118850 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e770681-34c0-45c2-b28d-719fd7117043 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:13.118850 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e770681-34c0-45c2-b28d-719fd7117043 None admin] PATH_INFO: `/v3/users/94a0cbc0f0d2429781903e722bf6eb53` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:13.134870 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e770681-34c0-45c2-b28d-719fd7117043 None admin] RBAC: Authorizing `identity:get_user(user_id=94a0cbc0f0d2429781903e722bf6eb53)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:13.136421 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e770681-34c0-45c2-b28d-719fd7117043 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:13.144799 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 96/194] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 24 06:21:12 2022] GET /identity/v3/users/94a0cbc0f0d2429781903e722bf6eb53 => generated 283 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:13.144799 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c06e930-9216-440b-af7b-6ead7e8df1a8 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:13.144799 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c06e930-9216-440b-af7b-6ead7e8df1a8 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:13.144799 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c06e930-9216-440b-af7b-6ead7e8df1a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:13.162182 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-9c06e930-9216-440b-af7b-6ead7e8df1a8 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:21:13.163419 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-9c06e930-9216-440b-af7b-6ead7e8df1a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:21:13.189346 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8facdd1a-96da-45be-84b7-b7dc9cfa363d None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:13.232341 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66d1dc9-853a-44bd-be00-37e1735fa979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:13.233394 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66d1dc9-853a-44bd-be00-37e1735fa979 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:13.234356 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66d1dc9-853a-44bd-be00-37e1735fa979 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:13.234654 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66d1dc9-853a-44bd-be00-37e1735fa979 None admin] PATH_INFO: `/v3/users/94a0cbc0f0d2429781903e722bf6eb53` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:13.238098 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66d1dc9-853a-44bd-be00-37e1735fa979 None admin] RBAC: Authorizing `identity:get_user(user_id=94a0cbc0f0d2429781903e722bf6eb53)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:13.239421 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66d1dc9-853a-44bd-be00-37e1735fa979 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:13.244428 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 99/195] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 24 06:21:13 2022] GET /identity/v3/users/94a0cbc0f0d2429781903e722bf6eb53 => generated 283 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:13.244428 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66d1dc9-853a-44bd-be00-37e1735fa979 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:13.267142 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 97/196] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:21:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:21:13.268025 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c06e930-9216-440b-af7b-6ead7e8df1a8 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:13.310389 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:13.394557 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66d1dc9-853a-44bd-be00-37e1735fa979 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:13.419476 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:13.430935 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c06e930-9216-440b-af7b-6ead7e8df1a8 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:13.448723 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062e86bf-2c58-4a6a-a43c-df3a26ba774b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:13.449945 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062e86bf-2c58-4a6a-a43c-df3a26ba774b None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:13.450414 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062e86bf-2c58-4a6a-a43c-df3a26ba774b None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:13.450867 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062e86bf-2c58-4a6a-a43c-df3a26ba774b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:13.487215 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca65a8f9-9097-42bf-a05a-be519f4f1136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:13.488735 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca65a8f9-9097-42bf-a05a-be519f4f1136 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:13.489165 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca65a8f9-9097-42bf-a05a-be519f4f1136 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:13.489561 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca65a8f9-9097-42bf-a05a-be519f4f1136 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:13.555834 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062e86bf-2c58-4a6a-a43c-df3a26ba774b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:13.557570 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062e86bf-2c58-4a6a-a43c-df3a26ba774b None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:13.585328 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca65a8f9-9097-42bf-a05a-be519f4f1136 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:13.587138 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca65a8f9-9097-42bf-a05a-be519f4f1136 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:13.630013 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:13.705988 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 100/197] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 24 06:21:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 06:21:13.707194 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062e86bf-2c58-4a6a-a43c-df3a26ba774b None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:13.725506 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:13.742409 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 98/198] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 24 06:21:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 06:21:13.748527 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca65a8f9-9097-42bf-a05a-be519f4f1136 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:13.838923 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062e86bf-2c58-4a6a-a43c-df3a26ba774b None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:13.892513 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531614f2-9106-428b-a71e-ef19decef344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:13.893726 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531614f2-9106-428b-a71e-ef19decef344 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:13.894217 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531614f2-9106-428b-a71e-ef19decef344 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:13.894563 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca65a8f9-9097-42bf-a05a-be519f4f1136 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:13.895448 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531614f2-9106-428b-a71e-ef19decef344 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:13.912532 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531614f2-9106-428b-a71e-ef19decef344 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:21:13.920022 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-531614f2-9106-428b-a71e-ef19decef344 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 06:21:13.920022 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:21:13.920022 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:21:13.920022 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:21:13.920022 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:21:13.920022 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:21:13.920022 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:21:13.920022 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:21:13.920022 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:21:13.920022 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:21:13.920022 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:21:13.920022 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:21:13.920022 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:21:13.920022 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:21:13.920022 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:21:13.920022 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:21:13.921114 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:21:13.921114 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:21:13.921114 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:21:13.921114 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:21:13.921114 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:21:13.921114 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:21:13.921114 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:21:13.921114 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:21:13.921114 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:21:13.921114 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:21:13.921114 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:21:13.921114 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:21:13.921114 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:21:13.921114 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:21:13.921114 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:21:13.921114 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:21:13.921982 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:21:13.921982 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:21:13.921982 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:21:13.921982 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:21:13.921982 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:21:13.921982 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:21:13.921982 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:21:13.921982 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:21:13.921982 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:21:13.921982 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:21:13.921982 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 06:21:13.921982 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:21:13.922586 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 101/199] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 24 06:21:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:21:13.923909 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531614f2-9106-428b-a71e-ef19decef344 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:13.943575 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:13.961008 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b68090-3d1d-486f-9398-7e6e1e1a592c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:13.961008 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b68090-3d1d-486f-9398-7e6e1e1a592c None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:13.961008 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b68090-3d1d-486f-9398-7e6e1e1a592c None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:13.961008 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b68090-3d1d-486f-9398-7e6e1e1a592c None admin] PATH_INFO: `/v3/projects/c2a0c48ef166451b912daa9e82ddcd09` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:13.961450 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b68090-3d1d-486f-9398-7e6e1e1a592c None admin] RBAC: Authorizing `identity:get_project(project_id=c2a0c48ef166451b912daa9e82ddcd09)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:13.961450 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b68090-3d1d-486f-9398-7e6e1e1a592c None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:13.962691 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 99/200] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 24 06:21:13 2022] GET /identity/v3/projects/c2a0c48ef166451b912daa9e82ddcd09 => generated 302 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:13.963607 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b68090-3d1d-486f-9398-7e6e1e1a592c None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:14.041496 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:14.080058 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531614f2-9106-428b-a71e-ef19decef344 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:14.089746 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b68090-3d1d-486f-9398-7e6e1e1a592c None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:14.129873 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff38808-fd37-43ea-9464-10ab24c99db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:14.130865 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff38808-fd37-43ea-9464-10ab24c99db9 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:14.131426 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff38808-fd37-43ea-9464-10ab24c99db9 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:14.132011 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff38808-fd37-43ea-9464-10ab24c99db9 None admin] PATH_INFO: `/v3/projects/c2a0c48ef166451b912daa9e82ddcd09` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:14.132850 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4c8a36-e9c5-4bd6-ae8b-8dad4913763b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:14.133610 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4c8a36-e9c5-4bd6-ae8b-8dad4913763b None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:14.133870 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff38808-fd37-43ea-9464-10ab24c99db9 None admin] RBAC: Authorizing `identity:get_project(project_id=c2a0c48ef166451b912daa9e82ddcd09)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:14.133978 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4c8a36-e9c5-4bd6-ae8b-8dad4913763b None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:14.134301 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4c8a36-e9c5-4bd6-ae8b-8dad4913763b None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:14.135057 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4c8a36-e9c5-4bd6-ae8b-8dad4913763b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:14.135487 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4c8a36-e9c5-4bd6-ae8b-8dad4913763b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:14.136454 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff38808-fd37-43ea-9464-10ab24c99db9 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:14.138047 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4c8a36-e9c5-4bd6-ae8b-8dad4913763b None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:14.138584 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 102/201] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 24 06:21:13 2022] GET /identity/v3/projects/c2a0c48ef166451b912daa9e82ddcd09 => generated 302 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:14.144207 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff38808-fd37-43ea-9464-10ab24c99db9 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:14.156145 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 100/202] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Aug 24 06:21:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:14.157815 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4c8a36-e9c5-4bd6-ae8b-8dad4913763b None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:14.244402 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:14.270960 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff38808-fd37-43ea-9464-10ab24c99db9 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:14.303815 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4c8a36-e9c5-4bd6-ae8b-8dad4913763b None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:14.318410 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a704a7e1-0fb8-4ee4-9269-e00997e2d02c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:14.319741 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a704a7e1-0fb8-4ee4-9269-e00997e2d02c None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:14.320259 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a704a7e1-0fb8-4ee4-9269-e00997e2d02c None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:14.320836 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a704a7e1-0fb8-4ee4-9269-e00997e2d02c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:14.321918 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a704a7e1-0fb8-4ee4-9269-e00997e2d02c None admin] RBAC: Adding query filter params (role.id=486144ad1bcd417e86acdbc1d14327bf, scope.project.id=c2a0c48ef166451b912daa9e82ddcd09, user.id=94a0cbc0f0d2429781903e722bf6eb53) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:14.322458 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a704a7e1-0fb8-4ee4-9269-e00997e2d02c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:14.324140 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a704a7e1-0fb8-4ee4-9269-e00997e2d02c None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:14.332352 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 103/203] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 24 06:21:14 2022] GET /identity/v3/role_assignments?user.id=94a0cbc0f0d2429781903e722bf6eb53&scope.project.id=c2a0c48ef166451b912daa9e82ddcd09&role.id=486144ad1bcd417e86acdbc1d14327bf => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:14.333660 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a704a7e1-0fb8-4ee4-9269-e00997e2d02c None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:14.346143 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:14.348190 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951ca33e-e17d-406a-8699-b1600ad88be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:14.348838 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951ca33e-e17d-406a-8699-b1600ad88be0 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:14.349121 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951ca33e-e17d-406a-8699-b1600ad88be0 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:14.349430 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951ca33e-e17d-406a-8699-b1600ad88be0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:14.350188 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951ca33e-e17d-406a-8699-b1600ad88be0 None admin] RBAC: Adding query filter params (role.id=b71c87e12dc1452ea3c99a7bf28585ca, scope.project.id=c2a0c48ef166451b912daa9e82ddcd09, user.id=94a0cbc0f0d2429781903e722bf6eb53) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:14.350502 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951ca33e-e17d-406a-8699-b1600ad88be0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:14.352436 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951ca33e-e17d-406a-8699-b1600ad88be0 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:14.360858 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 101/204] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 24 06:21:14 2022] GET /identity/v3/role_assignments?user.id=94a0cbc0f0d2429781903e722bf6eb53&scope.project.id=c2a0c48ef166451b912daa9e82ddcd09&role.id=b71c87e12dc1452ea3c99a7bf28585ca => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:14.592375 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a704a7e1-0fb8-4ee4-9269-e00997e2d02c None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:14.662994 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a19860-93c6-402b-b90e-15ce9adf7fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:14.663944 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a19860-93c6-402b-b90e-15ce9adf7fa3 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:14.664377 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a19860-93c6-402b-b90e-15ce9adf7fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:14.664799 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a19860-93c6-402b-b90e-15ce9adf7fa3 None admin] PATH_INFO: `/v3/users/f77fd3d9bc3c45bcafb43f716824579a` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:14.666258 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a19860-93c6-402b-b90e-15ce9adf7fa3 None admin] RBAC: Authorizing `identity:get_user(user_id=f77fd3d9bc3c45bcafb43f716824579a)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:14.668006 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a19860-93c6-402b-b90e-15ce9adf7fa3 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:14.669874 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 104/205] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 24 06:21:14 2022] GET /identity/v3/users/f77fd3d9bc3c45bcafb43f716824579a => generated 289 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:14.676007 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951ca33e-e17d-406a-8699-b1600ad88be0 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:14.808256 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951ca33e-e17d-406a-8699-b1600ad88be0 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:14.839785 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:14.847856 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24979cc9-264a-4b72-b108-3998384fd70a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:14.848563 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24979cc9-264a-4b72-b108-3998384fd70a None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:14.848953 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24979cc9-264a-4b72-b108-3998384fd70a None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:14.849372 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24979cc9-264a-4b72-b108-3998384fd70a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:14.926440 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24979cc9-264a-4b72-b108-3998384fd70a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:14.928107 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24979cc9-264a-4b72-b108-3998384fd70a None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:15.076401 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 102/206] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 24 06:21:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 06:21:15.083020 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a19860-93c6-402b-b90e-15ce9adf7fa3 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:15.281619 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:15.347476 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a19860-93c6-402b-b90e-15ce9adf7fa3 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:15.375403 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655fd961-0edc-40a6-9528-a7e90b6de77c None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:15.375951 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655fd961-0edc-40a6-9528-a7e90b6de77c None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:15.375951 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655fd961-0edc-40a6-9528-a7e90b6de77c None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:15.377592 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 103/207] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:21:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 06:21:15.383654 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060ee692-d793-4477-be9e-08e49ca2105c None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:15.383967 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060ee692-d793-4477-be9e-08e49ca2105c None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:15.384200 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060ee692-d793-4477-be9e-08e49ca2105c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:15.406943 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-060ee692-d793-4477-be9e-08e49ca2105c None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:21:15.408690 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-060ee692-d793-4477-be9e-08e49ca2105c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:21:15.418686 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9c9b10-ac46-4f17-8606-6c2ffdac84d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:15.419813 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9c9b10-ac46-4f17-8606-6c2ffdac84d1 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:15.420323 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9c9b10-ac46-4f17-8606-6c2ffdac84d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:15.421037 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9c9b10-ac46-4f17-8606-6c2ffdac84d1 None admin] PATH_INFO: `/v3/projects/df345274c3b840b486c303c6b301052e` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:15.422516 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9c9b10-ac46-4f17-8606-6c2ffdac84d1 None admin] RBAC: Authorizing `identity:get_project(project_id=df345274c3b840b486c303c6b301052e)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:15.423755 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:15.425226 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9c9b10-ac46-4f17-8606-6c2ffdac84d1 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:15.427359 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 105/208] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 24 06:21:15 2022] GET /identity/v3/projects/df345274c3b840b486c303c6b301052e => generated 298 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:15.433352 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9c9b10-ac46-4f17-8606-6c2ffdac84d1 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:15.492572 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 104/209] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:21:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 06:21:15.615339 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9c9b10-ac46-4f17-8606-6c2ffdac84d1 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:15.652614 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56d9dc9-8e3f-46f2-b6f8-c555ad2ddcbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:15.653245 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56d9dc9-8e3f-46f2-b6f8-c555ad2ddcbb None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:15.653527 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56d9dc9-8e3f-46f2-b6f8-c555ad2ddcbb None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:15.653844 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56d9dc9-8e3f-46f2-b6f8-c555ad2ddcbb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:15.654586 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56d9dc9-8e3f-46f2-b6f8-c555ad2ddcbb None admin] RBAC: Adding query filter params (role.id=fcfa05f1d2b1474bb4fc20fc988cacd9, scope.project.id=df345274c3b840b486c303c6b301052e, user.id=f77fd3d9bc3c45bcafb43f716824579a) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:15.654890 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56d9dc9-8e3f-46f2-b6f8-c555ad2ddcbb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:15.656518 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56d9dc9-8e3f-46f2-b6f8-c555ad2ddcbb None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:15.663445 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 106/210] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 24 06:21:15 2022] GET /identity/v3/role_assignments?user.id=f77fd3d9bc3c45bcafb43f716824579a&scope.project.id=df345274c3b840b486c303c6b301052e&role.id=fcfa05f1d2b1474bb4fc20fc988cacd9 => generated 608 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:21:15.826625 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10206b49-91b0-4c6b-bb4f-109fb9027519 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:15.827520 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10206b49-91b0-4c6b-bb4f-109fb9027519 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:15.827793 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10206b49-91b0-4c6b-bb4f-109fb9027519 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:15.829780 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 105/211] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:21:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:21:15.843446 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10206b49-91b0-4c6b-bb4f-109fb9027519 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:15.847996 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30bf4ff-6869-49a3-93f2-cee737e7e147 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:15.849498 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30bf4ff-6869-49a3-93f2-cee737e7e147 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:15.849899 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30bf4ff-6869-49a3-93f2-cee737e7e147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:15.876763 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:15.883777 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-c30bf4ff-6869-49a3-93f2-cee737e7e147 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:21:15.886435 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-c30bf4ff-6869-49a3-93f2-cee737e7e147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:21:15.976123 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 107/212] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:21:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:21:16.018221 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10206b49-91b0-4c6b-bb4f-109fb9027519 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:16.054480 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3829a16-45f9-4783-a927-8561cd91485b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:16.055138 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3829a16-45f9-4783-a927-8561cd91485b None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:16.055416 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3829a16-45f9-4783-a927-8561cd91485b None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:16.055688 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3829a16-45f9-4783-a927-8561cd91485b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:16.067570 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3829a16-45f9-4783-a927-8561cd91485b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:21:16.072367 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-d3829a16-45f9-4783-a927-8561cd91485b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 06:21:16.072367 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:21:16.072367 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:21:16.072367 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:21:16.072367 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:21:16.072367 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:21:16.072367 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:21:16.072367 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:21:16.072367 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:21:16.072367 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:21:16.072367 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:21:16.072367 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:21:16.072367 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:21:16.072367 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:21:16.072367 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:21:16.072367 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:21:16.073269 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:21:16.073269 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:21:16.073269 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:21:16.073269 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:21:16.073269 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:21:16.073269 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:21:16.073269 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:21:16.073269 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:21:16.073269 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:21:16.073269 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:21:16.073269 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:21:16.073269 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:21:16.073269 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:21:16.073269 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:21:16.073269 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:21:16.073269 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:21:16.074065 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:21:16.074065 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:21:16.074065 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:21:16.074065 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:21:16.074065 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:21:16.074065 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:21:16.074065 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:21:16.074065 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:21:16.074065 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:21:16.074065 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:21:16.074065 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 06:21:16.074065 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:21:16.074065 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 106/213] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 24 06:21:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:21:16.081645 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30bf4ff-6869-49a3-93f2-cee737e7e147 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:16.197489 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30bf4ff-6869-49a3-93f2-cee737e7e147 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:16.237533 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f429b2ae-1bbc-4644-88f8-2f1df426b993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:16.238469 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f429b2ae-1bbc-4644-88f8-2f1df426b993 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:16.238924 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f429b2ae-1bbc-4644-88f8-2f1df426b993 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:16.239384 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f429b2ae-1bbc-4644-88f8-2f1df426b993 None admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:16.240475 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f429b2ae-1bbc-4644-88f8-2f1df426b993 None admin] RBAC: Adding query filter params (name=member) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:16.240954 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f429b2ae-1bbc-4644-88f8-2f1df426b993 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:16.242461 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f429b2ae-1bbc-4644-88f8-2f1df426b993 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:16.245031 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:16.252239 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 108/214] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Aug 24 06:21:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:16.257519 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3829a16-45f9-4783-a927-8561cd91485b None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:16.270201 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:16.316278 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f429b2ae-1bbc-4644-88f8-2f1df426b993 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:16.370070 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3829a16-45f9-4783-a927-8561cd91485b None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:16.408975 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce58137-3805-4759-a8a7-12e1e2dd6912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:16.409653 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce58137-3805-4759-a8a7-12e1e2dd6912 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:16.409917 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce58137-3805-4759-a8a7-12e1e2dd6912 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:16.410467 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce58137-3805-4759-a8a7-12e1e2dd6912 None admin] PATH_INFO: `/v3/users/a15565acd41d402aaabba3fc11ec9e7a` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:16.426513 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce58137-3805-4759-a8a7-12e1e2dd6912 None admin] RBAC: Authorizing `identity:get_user(user_id=a15565acd41d402aaabba3fc11ec9e7a)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:16.428205 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce58137-3805-4759-a8a7-12e1e2dd6912 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:16.430109 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 107/215] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 24 06:21:16 2022] GET /identity/v3/users/a15565acd41d402aaabba3fc11ec9e7a => generated 297 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:16.436235 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce58137-3805-4759-a8a7-12e1e2dd6912 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:16.457423 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f429b2ae-1bbc-4644-88f8-2f1df426b993 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:16.498293 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8dc5f6-16e7-47ac-8591-751128acdf13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:16.499643 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8dc5f6-16e7-47ac-8591-751128acdf13 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:16.500029 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8dc5f6-16e7-47ac-8591-751128acdf13 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:16.500399 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8dc5f6-16e7-47ac-8591-751128acdf13 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:16.501871 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8dc5f6-16e7-47ac-8591-751128acdf13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:16.503465 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8dc5f6-16e7-47ac-8591-751128acdf13 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:16.505206 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 109/216] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:21:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:21:16.511081 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8dc5f6-16e7-47ac-8591-751128acdf13 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:16.557585 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:16.595683 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:16.598183 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce58137-3805-4759-a8a7-12e1e2dd6912 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:16.654659 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c725d9cc-bc3e-4909-9a95-f3cc06098f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:16.655904 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c725d9cc-bc3e-4909-9a95-f3cc06098f78 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:16.656421 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c725d9cc-bc3e-4909-9a95-f3cc06098f78 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:16.656748 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c725d9cc-bc3e-4909-9a95-f3cc06098f78 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:16.659796 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8dc5f6-16e7-47ac-8591-751128acdf13 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:16.718799 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f0da7e-632e-48a4-b0ea-9df21cf09f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:16.719541 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f0da7e-632e-48a4-b0ea-9df21cf09f90 None admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:16.719899 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f0da7e-632e-48a4-b0ea-9df21cf09f90 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:16.720178 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f0da7e-632e-48a4-b0ea-9df21cf09f90 None admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:16.720912 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f0da7e-632e-48a4-b0ea-9df21cf09f90 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:16.722423 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f0da7e-632e-48a4-b0ea-9df21cf09f90 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:16.747856 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c725d9cc-bc3e-4909-9a95-f3cc06098f78 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:16.756561 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c725d9cc-bc3e-4909-9a95-f3cc06098f78 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:16.832338 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:16.871988 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 108/217] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 24 06:21:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 06:21:16.877598 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c725d9cc-bc3e-4909-9a95-f3cc06098f78 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:16.912433 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 110/218] 10.222.0.47 () {60 vars in 1132 bytes} [Wed Aug 24 06:21:16 2022] POST /identity/v3/users => generated 297 bytes in 402 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:21:16.915774 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fbeb26-d71d-4185-841d-ac34fc739e5e None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:16.916078 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fbeb26-d71d-4185-841d-ac34fc739e5e None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:16.916333 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fbeb26-d71d-4185-841d-ac34fc739e5e None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:16.918132 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 111/219] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:21:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 06:21:16.924866 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68edebf-b205-46d5-9ee4-8fdb37d0da93 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:16.925205 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68edebf-b205-46d5-9ee4-8fdb37d0da93 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:16.925449 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68edebf-b205-46d5-9ee4-8fdb37d0da93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:16.945799 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-b68edebf-b205-46d5-9ee4-8fdb37d0da93 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:21:16.947050 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-b68edebf-b205-46d5-9ee4-8fdb37d0da93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:21:17.026977 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c725d9cc-bc3e-4909-9a95-f3cc06098f78 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:17.051378 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 112/220] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:21:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 06:21:17.072730 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dc0421-5e42-4ec8-85b9-f2aefcdd1300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:17.073410 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dc0421-5e42-4ec8-85b9-f2aefcdd1300 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:17.073693 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dc0421-5e42-4ec8-85b9-f2aefcdd1300 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:17.073964 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dc0421-5e42-4ec8-85b9-f2aefcdd1300 None admin] PATH_INFO: `/v3/projects/c2a0c48ef166451b912daa9e82ddcd09` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:17.075134 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dc0421-5e42-4ec8-85b9-f2aefcdd1300 None admin] RBAC: Authorizing `identity:get_project(project_id=c2a0c48ef166451b912daa9e82ddcd09)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:17.076599 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dc0421-5e42-4ec8-85b9-f2aefcdd1300 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:17.079033 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 109/221] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 24 06:21:16 2022] GET /identity/v3/projects/c2a0c48ef166451b912daa9e82ddcd09 => generated 302 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:17.085215 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68edebf-b205-46d5-9ee4-8fdb37d0da93 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:17.091013 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b8bda5-b9e3-45ff-a41e-dd48a00dcc08 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:17.091294 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b8bda5-b9e3-45ff-a41e-dd48a00dcc08 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:17.091600 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b8bda5-b9e3-45ff-a41e-dd48a00dcc08 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:17.093698 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 110/222] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:21:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 06:21:17.099952 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8c4cae-95c5-40de-be7a-ec687a0e58d3 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:17.100219 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8c4cae-95c5-40de-be7a-ec687a0e58d3 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:17.100546 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8c4cae-95c5-40de-be7a-ec687a0e58d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:17.122726 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-fb8c4cae-95c5-40de-be7a-ec687a0e58d3 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:21:17.124801 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-fb8c4cae-95c5-40de-be7a-ec687a0e58d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:21:17.159579 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:17.169423 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:17.237578 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68edebf-b205-46d5-9ee4-8fdb37d0da93 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:17.243237 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 111/223] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:21:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:21:17.284380 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ce1e84-6ab1-43a1-85b6-2865cd070de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:17.284987 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ce1e84-6ab1-43a1-85b6-2865cd070de2 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:17.285295 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ce1e84-6ab1-43a1-85b6-2865cd070de2 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:17.285570 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ce1e84-6ab1-43a1-85b6-2865cd070de2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:17.286411 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ce1e84-6ab1-43a1-85b6-2865cd070de2 None admin] RBAC: Adding query filter params (role.id=e1f9b167c51c4a388cbc9594db8a465d, scope.project.id=c2a0c48ef166451b912daa9e82ddcd09, user.id=a15565acd41d402aaabba3fc11ec9e7a) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:17.286689 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ce1e84-6ab1-43a1-85b6-2865cd070de2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:17.288276 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ce1e84-6ab1-43a1-85b6-2865cd070de2 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:17.298074 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 113/224] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 24 06:21:17 2022] GET /identity/v3/role_assignments?user.id=a15565acd41d402aaabba3fc11ec9e7a&scope.project.id=c2a0c48ef166451b912daa9e82ddcd09&role.id=e1f9b167c51c4a388cbc9594db8a465d => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:17.342385 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8c4cae-95c5-40de-be7a-ec687a0e58d3 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:17.472620 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8c4cae-95c5-40de-be7a-ec687a0e58d3 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:17.509999 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5371d0-b587-47bc-a0a0-b35f565c43b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:17.510656 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5371d0-b587-47bc-a0a0-b35f565c43b2 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:17.510937 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5371d0-b587-47bc-a0a0-b35f565c43b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:17.511212 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5371d0-b587-47bc-a0a0-b35f565c43b2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:17.522719 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5371d0-b587-47bc-a0a0-b35f565c43b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:21:17.527959 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-ad5371d0-b587-47bc-a0a0-b35f565c43b2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:21:17.527959 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:21:17.527959 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:21:17.527959 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:21:17.527959 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:21:17.527959 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:21:17.527959 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:21:17.527959 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:21:17.527959 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:21:17.527959 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:21:17.527959 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:21:17.527959 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:21:17.527959 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:21:17.527959 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:21:17.527959 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:21:17.527959 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:21:17.528836 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:21:17.528836 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:21:17.528836 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:21:17.528836 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:21:17.528836 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:21:17.528836 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:21:17.528836 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:21:17.528836 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:21:17.528836 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:21:17.528836 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:21:17.528836 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:21:17.528836 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:21:17.528836 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:21:17.528836 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:21:17.528836 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:21:17.528836 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:21:17.529770 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:21:17.529770 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:21:17.529770 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:21:17.529770 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:21:17.529770 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:21:17.529770 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:21:17.529770 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:21:17.529770 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:21:17.529770 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:21:17.529770 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:21:17.529770 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:21:17.529770 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:21:17.529770 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 112/225] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Aug 24 06:21:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:21:17.538659 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ce1e84-6ab1-43a1-85b6-2865cd070de2 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:17.572616 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5371d0-b587-47bc-a0a0-b35f565c43b2 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:17.606108 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:17.709436 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ce1e84-6ab1-43a1-85b6-2865cd070de2 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:17.729155 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:17.758970 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65160c8-58b0-4054-8019-f0a1539c0632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:17.759888 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65160c8-58b0-4054-8019-f0a1539c0632 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:17.760348 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65160c8-58b0-4054-8019-f0a1539c0632 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:17.760784 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65160c8-58b0-4054-8019-f0a1539c0632 None admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:17.761508 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65160c8-58b0-4054-8019-f0a1539c0632 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:17.761815 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65160c8-58b0-4054-8019-f0a1539c0632 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:17.763233 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65160c8-58b0-4054-8019-f0a1539c0632 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:17.766247 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5371d0-b587-47bc-a0a0-b35f565c43b2 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:17.783349 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 114/226] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 24 06:21:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:17.789908 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65160c8-58b0-4054-8019-f0a1539c0632 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:17.842325 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088e7912-92d9-4f90-ab55-47f9d5783764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:17.843403 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088e7912-92d9-4f90-ab55-47f9d5783764 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:17.843946 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088e7912-92d9-4f90-ab55-47f9d5783764 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:17.844409 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088e7912-92d9-4f90-ab55-47f9d5783764 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:17.872034 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088e7912-92d9-4f90-ab55-47f9d5783764 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:21:17.877682 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-088e7912-92d9-4f90-ab55-47f9d5783764 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 06:21:17.877682 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:21:17.877682 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:21:17.877682 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:21:17.877682 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:21:17.877682 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:21:17.877682 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:21:17.877682 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:21:17.877682 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:21:17.877682 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:21:17.877682 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:21:17.877682 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:21:17.877682 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:21:17.877682 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:21:17.877682 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:21:17.877682 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:21:17.878603 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:21:17.878603 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:21:17.878603 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:21:17.878603 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:21:17.878603 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:21:17.878603 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:21:17.878603 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:21:17.878603 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:21:17.878603 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:21:17.878603 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:21:17.878603 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:21:17.878603 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:21:17.878603 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:21:17.878603 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:21:17.878603 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:21:17.878603 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:21:17.879492 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:21:17.879492 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:21:17.879492 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:21:17.879492 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:21:17.879492 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:21:17.879492 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:21:17.879492 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:21:17.879492 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:21:17.879492 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:21:17.879492 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:21:17.879492 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 06:21:17.879492 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:21:17.879492 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 113/227] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Aug 24 06:21:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:21:17.889370 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088e7912-92d9-4f90-ab55-47f9d5783764 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:17.949261 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65160c8-58b0-4054-8019-f0a1539c0632 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:17.987895 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80525612-0428-4ae3-b5ac-45143e5e7101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:17.988891 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80525612-0428-4ae3-b5ac-45143e5e7101 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:17.989425 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80525612-0428-4ae3-b5ac-45143e5e7101 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:17.989923 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80525612-0428-4ae3-b5ac-45143e5e7101 None admin] PATH_INFO: `/v3/users/94a0cbc0f0d2429781903e722bf6eb53` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:17.991313 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80525612-0428-4ae3-b5ac-45143e5e7101 None admin] RBAC: Authorizing `identity:get_user(user_id=94a0cbc0f0d2429781903e722bf6eb53)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:17.992950 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80525612-0428-4ae3-b5ac-45143e5e7101 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:17.995211 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 115/228] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 24 06:21:17 2022] GET /identity/v3/users/94a0cbc0f0d2429781903e722bf6eb53 => generated 283 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:21:18.001971 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80525612-0428-4ae3-b5ac-45143e5e7101 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:18.054819 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:18.076006 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:18.114598 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80525612-0428-4ae3-b5ac-45143e5e7101 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:18.118084 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088e7912-92d9-4f90-ab55-47f9d5783764 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:18.154368 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7203db99-80d7-467e-a9f8-914216ff708a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:18.155925 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7203db99-80d7-467e-a9f8-914216ff708a None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:18.156400 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7203db99-80d7-467e-a9f8-914216ff708a None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:18.156865 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7203db99-80d7-467e-a9f8-914216ff708a None admin] PATH_INFO: `/v3/projects/c2a0c48ef166451b912daa9e82ddcd09` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:18.158263 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7203db99-80d7-467e-a9f8-914216ff708a None admin] RBAC: Authorizing `identity:get_project(project_id=c2a0c48ef166451b912daa9e82ddcd09)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:18.160088 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7203db99-80d7-467e-a9f8-914216ff708a None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:18.162583 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 116/229] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 24 06:21:18 2022] GET /identity/v3/projects/c2a0c48ef166451b912daa9e82ddcd09 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:18.168462 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7203db99-80d7-467e-a9f8-914216ff708a None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:18.181931 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abf4a5f-d91e-48e9-82f3-b920f509acf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:18.182566 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abf4a5f-d91e-48e9-82f3-b920f509acf7 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:18.182845 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abf4a5f-d91e-48e9-82f3-b920f509acf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:18.183113 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abf4a5f-d91e-48e9-82f3-b920f509acf7 None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:18.184072 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abf4a5f-d91e-48e9-82f3-b920f509acf7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:18.184072 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abf4a5f-d91e-48e9-82f3-b920f509acf7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:18.185501 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abf4a5f-d91e-48e9-82f3-b920f509acf7 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:18.201222 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 114/230] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 24 06:21:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:18.210254 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abf4a5f-d91e-48e9-82f3-b920f509acf7 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:18.282933 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7203db99-80d7-467e-a9f8-914216ff708a None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:18.318778 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:18.326041 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8661a163-e1db-43f4-aa19-240bb6d63938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:18.327166 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8661a163-e1db-43f4-aa19-240bb6d63938 None admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:18.327664 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8661a163-e1db-43f4-aa19-240bb6d63938 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:18.328127 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8661a163-e1db-43f4-aa19-240bb6d63938 None admin] PATH_INFO: `/v3/projects/c2a0c48ef166451b912daa9e82ddcd09/users/94a0cbc0f0d2429781903e722bf6eb53/roles/486144ad1bcd417e86acdbc1d14327bf` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:18.330748 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8661a163-e1db-43f4-aa19-240bb6d63938 None admin] RBAC: Authorizing `identity:create_grant(project_id=c2a0c48ef166451b912daa9e82ddcd09, user_id=94a0cbc0f0d2429781903e722bf6eb53, role_id=486144ad1bcd417e86acdbc1d14327bf)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:18.334270 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8661a163-e1db-43f4-aa19-240bb6d63938 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:18.398833 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abf4a5f-d91e-48e9-82f3-b920f509acf7 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:18.407907 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 117/231] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Aug 24 06:21:18 2022] PUT /identity/v3/projects/c2a0c48ef166451b912daa9e82ddcd09/users/94a0cbc0f0d2429781903e722bf6eb53/roles/486144ad1bcd417e86acdbc1d14327bf => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:21:18.456185 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8aee666-b216-400a-901c-1f0cdce18c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:18.457243 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aee666-b216-400a-901c-1f0cdce18c0a None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:18.457701 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aee666-b216-400a-901c-1f0cdce18c0a None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:18.458147 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aee666-b216-400a-901c-1f0cdce18c0a None admin] PATH_INFO: `/v3/users/94a0cbc0f0d2429781903e722bf6eb53` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:18.459711 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8aee666-b216-400a-901c-1f0cdce18c0a None admin] RBAC: Authorizing `identity:get_user(user_id=94a0cbc0f0d2429781903e722bf6eb53)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:18.461442 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8aee666-b216-400a-901c-1f0cdce18c0a None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:18.463783 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 115/232] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 24 06:21:18 2022] GET /identity/v3/users/94a0cbc0f0d2429781903e722bf6eb53 => generated 283 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:18.471963 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8661a163-e1db-43f4-aa19-240bb6d63938 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:18.645357 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8661a163-e1db-43f4-aa19-240bb6d63938 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:18.687537 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2c0124-697f-4d14-98e2-95da342e3d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:18.688620 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2c0124-697f-4d14-98e2-95da342e3d7c None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:18.689073 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2c0124-697f-4d14-98e2-95da342e3d7c None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:18.689544 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2c0124-697f-4d14-98e2-95da342e3d7c None admin] PATH_INFO: `/v3/projects/c2a0c48ef166451b912daa9e82ddcd09` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:18.691021 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2c0124-697f-4d14-98e2-95da342e3d7c None admin] RBAC: Authorizing `identity:get_project(project_id=c2a0c48ef166451b912daa9e82ddcd09)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:18.692640 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2c0124-697f-4d14-98e2-95da342e3d7c None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:18.694844 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 118/233] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 24 06:21:18 2022] GET /identity/v3/projects/c2a0c48ef166451b912daa9e82ddcd09 => generated 302 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:18.701430 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8aee666-b216-400a-901c-1f0cdce18c0a None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:18.722775 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:18.845273 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8aee666-b216-400a-901c-1f0cdce18c0a None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:18.893767 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ea662b-b654-4866-a644-aca1f270a291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:18.894727 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ea662b-b654-4866-a644-aca1f270a291 None admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:18.895182 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ea662b-b654-4866-a644-aca1f270a291 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:18.895629 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ea662b-b654-4866-a644-aca1f270a291 None admin] PATH_INFO: `/v3/projects/c2a0c48ef166451b912daa9e82ddcd09/users/94a0cbc0f0d2429781903e722bf6eb53/roles/b71c87e12dc1452ea3c99a7bf28585ca` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:18.898092 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ea662b-b654-4866-a644-aca1f270a291 None admin] RBAC: Authorizing `identity:create_grant(project_id=c2a0c48ef166451b912daa9e82ddcd09, user_id=94a0cbc0f0d2429781903e722bf6eb53, role_id=b71c87e12dc1452ea3c99a7bf28585ca)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:18.899730 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ea662b-b654-4866-a644-aca1f270a291 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:19.041818 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 116/234] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Aug 24 06:21:18 2022] PUT /identity/v3/projects/c2a0c48ef166451b912daa9e82ddcd09/users/94a0cbc0f0d2429781903e722bf6eb53/roles/b71c87e12dc1452ea3c99a7bf28585ca => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:21:19.405671 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d26f310-12ea-4777-838e-bc3cd433191c None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:19.406933 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d26f310-12ea-4777-838e-bc3cd433191c None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:19.407290 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d26f310-12ea-4777-838e-bc3cd433191c None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:19.408871 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 119/235] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:21:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:21:19.414811 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f341312-4069-416b-93d0-b9b16a9a0f7d None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:19.415388 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f341312-4069-416b-93d0-b9b16a9a0f7d None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:19.415904 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f341312-4069-416b-93d0-b9b16a9a0f7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:19.436139 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-7f341312-4069-416b-93d0-b9b16a9a0f7d None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:21:19.437750 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-7f341312-4069-416b-93d0-b9b16a9a0f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:21:19.519013 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 117/236] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:21:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:21:19.713457 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c9170d-d19c-41be-9eb8-ee5176772338 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:19.713666 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c9170d-d19c-41be-9eb8-ee5176772338 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:19.713941 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c9170d-d19c-41be-9eb8-ee5176772338 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:19.715724 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 120/237] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:21:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:21:19.722441 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d130519-180d-471f-b103-42a0155fd24e None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:19.723196 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d130519-180d-471f-b103-42a0155fd24e None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:19.723196 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d130519-180d-471f-b103-42a0155fd24e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:19.731563 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:19.744625 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-3d130519-180d-471f-b103-42a0155fd24e None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:21:19.745950 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-3d130519-180d-471f-b103-42a0155fd24e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:21:19.773174 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d697e32c-0050-4111-af51-ae86bae4c748 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:19.773483 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d697e32c-0050-4111-af51-ae86bae4c748 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:19.773764 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d697e32c-0050-4111-af51-ae86bae4c748 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:19.775440 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 121/238] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:21:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:21:19.791068 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63000273-a415-4288-8c87-16b19a22bd5c None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:19.791462 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63000273-a415-4288-8c87-16b19a22bd5c None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:19.791612 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63000273-a415-4288-8c87-16b19a22bd5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:19.822392 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-63000273-a415-4288-8c87-16b19a22bd5c None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:21:19.823607 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-63000273-a415-4288-8c87-16b19a22bd5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:21:19.854983 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 118/239] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:21:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:21:19.891345 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:19.898654 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d130519-180d-471f-b103-42a0155fd24e None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:19.945545 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 122/240] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:21:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 06:21:20.033852 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d130519-180d-471f-b103-42a0155fd24e None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:20.072647 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b69f549-d6e1-4a2a-980d-cb5f562a5c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:20.073316 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b69f549-d6e1-4a2a-980d-cb5f562a5c5a None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:20.073594 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b69f549-d6e1-4a2a-980d-cb5f562a5c5a None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:20.073865 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b69f549-d6e1-4a2a-980d-cb5f562a5c5a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:20.075874 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:20.088756 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b69f549-d6e1-4a2a-980d-cb5f562a5c5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:21:20.093596 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-6b69f549-d6e1-4a2a-980d-cb5f562a5c5a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 06:21:20.093596 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:21:20.093596 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:21:20.093596 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:21:20.093596 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:21:20.093596 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:21:20.093596 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:21:20.093596 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:21:20.093596 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:21:20.093596 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:21:20.093596 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:21:20.093596 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:21:20.093596 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:21:20.093596 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:21:20.093596 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:21:20.093596 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:21:20.095145 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:21:20.095145 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:21:20.095145 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:21:20.095145 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:21:20.095145 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:21:20.095145 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:21:20.095145 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:21:20.095145 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:21:20.095145 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:21:20.095145 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:21:20.095145 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:21:20.095145 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:21:20.095145 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:21:20.095145 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:21:20.095145 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:21:20.095145 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:21:20.096004 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:21:20.096004 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:21:20.096004 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:21:20.096004 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:21:20.096004 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:21:20.096004 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:21:20.096004 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:21:20.096004 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:21:20.096004 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:21:20.096004 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:21:20.096004 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 06:21:20.096004 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:21:20.096004 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 119/241] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 24 06:21:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:21:20.102654 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63000273-a415-4288-8c87-16b19a22bd5c None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:20.278293 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b69f549-d6e1-4a2a-980d-cb5f562a5c5a None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:20.285661 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63000273-a415-4288-8c87-16b19a22bd5c None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:20.335838 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af08f824-2358-4ba8-87f3-90b4703d9d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:20.337215 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af08f824-2358-4ba8-87f3-90b4703d9d83 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:20.337733 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af08f824-2358-4ba8-87f3-90b4703d9d83 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:20.338220 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af08f824-2358-4ba8-87f3-90b4703d9d83 None admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:20.339709 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af08f824-2358-4ba8-87f3-90b4703d9d83 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:20.340222 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af08f824-2358-4ba8-87f3-90b4703d9d83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:20.342447 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af08f824-2358-4ba8-87f3-90b4703d9d83 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:20.354853 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 123/242] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Aug 24 06:21:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:20.356270 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af08f824-2358-4ba8-87f3-90b4703d9d83 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:20.392384 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b69f549-d6e1-4a2a-980d-cb5f562a5c5a None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:20.403954 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:20.432327 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2af36c-fe68-4d26-ada4-6e75be01ab9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:20.433261 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2af36c-fe68-4d26-ada4-6e75be01ab9e None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:20.433713 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2af36c-fe68-4d26-ada4-6e75be01ab9e None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:20.434149 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2af36c-fe68-4d26-ada4-6e75be01ab9e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:20.435444 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2af36c-fe68-4d26-ada4-6e75be01ab9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:20.437606 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2af36c-fe68-4d26-ada4-6e75be01ab9e None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:20.439767 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 120/243] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:21:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:20.441421 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2af36c-fe68-4d26-ada4-6e75be01ab9e None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:20.494142 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af08f824-2358-4ba8-87f3-90b4703d9d83 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:20.522202 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:20.551056 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ebbd00-9399-4dc3-b053-8db52d4628a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:20.551928 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ebbd00-9399-4dc3-b053-8db52d4628a1 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:20.552369 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ebbd00-9399-4dc3-b053-8db52d4628a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:20.552784 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ebbd00-9399-4dc3-b053-8db52d4628a1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:20.573061 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2af36c-fe68-4d26-ada4-6e75be01ab9e None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:20.578114 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ebbd00-9399-4dc3-b053-8db52d4628a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:21:20.588419 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-02ebbd00-9399-4dc3-b053-8db52d4628a1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 06:21:20.588419 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:21:20.588419 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:21:20.588419 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:21:20.588419 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:21:20.588419 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:21:20.588419 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:21:20.588419 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:21:20.588419 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:21:20.588419 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:21:20.588419 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:21:20.588419 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:21:20.588419 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:21:20.588419 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:21:20.588419 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:21:20.588419 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:21:20.589670 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:21:20.589670 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:21:20.589670 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:21:20.589670 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:21:20.589670 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:21:20.589670 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:21:20.589670 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:21:20.589670 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:21:20.589670 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:21:20.589670 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:21:20.589670 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:21:20.589670 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:21:20.589670 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:21:20.589670 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:21:20.589670 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:21:20.589670 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:21:20.590474 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:21:20.590474 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:21:20.590474 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:21:20.590474 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:21:20.590474 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:21:20.590474 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:21:20.590474 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:21:20.590474 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:21:20.590474 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:21:20.590474 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:21:20.590474 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 06:21:20.590474 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:21:20.591097 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 124/244] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 24 06:21:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:21:20.596279 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ebbd00-9399-4dc3-b053-8db52d4628a1 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:20.632412 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa89723-2a30-4880-acbd-71794d16a75f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:20.633572 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa89723-2a30-4880-acbd-71794d16a75f None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:20.634058 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa89723-2a30-4880-acbd-71794d16a75f None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:20.634536 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa89723-2a30-4880-acbd-71794d16a75f None admin] PATH_INFO: `/v3/users/360d2cdeb8fb48b89b7ac189977b0daa` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:20.653010 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa89723-2a30-4880-acbd-71794d16a75f None admin] RBAC: Authorizing `identity:get_user(user_id=360d2cdeb8fb48b89b7ac189977b0daa)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:20.654726 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa89723-2a30-4880-acbd-71794d16a75f None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:20.656584 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 121/245] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 24 06:21:20 2022] GET /identity/v3/users/360d2cdeb8fb48b89b7ac189977b0daa => generated 297 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:20.661997 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa89723-2a30-4880-acbd-71794d16a75f None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:20.743244 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:20.754918 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ebbd00-9399-4dc3-b053-8db52d4628a1 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:20.781996 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa89723-2a30-4880-acbd-71794d16a75f None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:20.804348 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b840c6a-1265-4a9a-8abc-515a711ef4ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:20.804993 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b840c6a-1265-4a9a-8abc-515a711ef4ce None admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:20.805415 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b840c6a-1265-4a9a-8abc-515a711ef4ce None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:20.805757 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b840c6a-1265-4a9a-8abc-515a711ef4ce None admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:20.806466 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b840c6a-1265-4a9a-8abc-515a711ef4ce None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:20.808008 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b840c6a-1265-4a9a-8abc-515a711ef4ce None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:20.814806 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:20.832384 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7990b27-8e65-498e-924e-1af1574a26f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:20.833156 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7990b27-8e65-498e-924e-1af1574a26f7 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:20.833853 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7990b27-8e65-498e-924e-1af1574a26f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:20.834065 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7990b27-8e65-498e-924e-1af1574a26f7 None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:20.834842 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7990b27-8e65-498e-924e-1af1574a26f7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:20.835297 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7990b27-8e65-498e-924e-1af1574a26f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:20.836777 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7990b27-8e65-498e-924e-1af1574a26f7 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:20.846394 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 122/246] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Aug 24 06:21:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:20.847477 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7990b27-8e65-498e-924e-1af1574a26f7 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:20.905164 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 125/247] 10.222.0.47 () {60 vars in 1132 bytes} [Wed Aug 24 06:21:20 2022] POST /identity/v3/users => generated 293 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:21:20.906069 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b840c6a-1265-4a9a-8abc-515a711ef4ce None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:20.976905 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7990b27-8e65-498e-924e-1af1574a26f7 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:21.017475 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44ac6e2-5da7-4bb3-a09d-d5aa31b98f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:21.019246 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44ac6e2-5da7-4bb3-a09d-d5aa31b98f04 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:21.019536 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44ac6e2-5da7-4bb3-a09d-d5aa31b98f04 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:21.019817 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44ac6e2-5da7-4bb3-a09d-d5aa31b98f04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:21.062338 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b840c6a-1265-4a9a-8abc-515a711ef4ce None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:21.097444 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:21.112561 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4e814b-708a-4bee-be3a-cc73bbfcf1f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:21.113248 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4e814b-708a-4bee-be3a-cc73bbfcf1f3 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:21.113536 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4e814b-708a-4bee-be3a-cc73bbfcf1f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:21.113809 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4e814b-708a-4bee-be3a-cc73bbfcf1f3 None admin] PATH_INFO: `/v3/users/a15565acd41d402aaabba3fc11ec9e7a` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:21.115108 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4e814b-708a-4bee-be3a-cc73bbfcf1f3 None admin] RBAC: Authorizing `identity:get_user(user_id=a15565acd41d402aaabba3fc11ec9e7a)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:21.118132 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4e814b-708a-4bee-be3a-cc73bbfcf1f3 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:21.120273 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 126/248] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 24 06:21:20 2022] GET /identity/v3/users/a15565acd41d402aaabba3fc11ec9e7a => generated 297 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:21.126579 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00c989a-f6f5-4ceb-9257-d2daeebca735 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:21.126579 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00c989a-f6f5-4ceb-9257-d2daeebca735 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:21.127112 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00c989a-f6f5-4ceb-9257-d2daeebca735 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:21.129846 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 127/249] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:21:21 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:21:21.133214 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00c989a-f6f5-4ceb-9257-d2daeebca735 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:21.133370 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44ac6e2-5da7-4bb3-a09d-d5aa31b98f04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:21.134452 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44ac6e2-5da7-4bb3-a09d-d5aa31b98f04 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:21.159350 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:21.283545 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 123/250] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 24 06:21:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 06:21:21.284950 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00c989a-f6f5-4ceb-9257-d2daeebca735 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:21.285163 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa24c920-6d3c-47a1-9d40-6fe4b7da8885 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:21.285498 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa24c920-6d3c-47a1-9d40-6fe4b7da8885 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:21.290257 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa24c920-6d3c-47a1-9d40-6fe4b7da8885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:21.312459 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-aa24c920-6d3c-47a1-9d40-6fe4b7da8885 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:21:21.313609 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-aa24c920-6d3c-47a1-9d40-6fe4b7da8885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:21:21.329018 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9481ccdd-2b4d-49aa-a266-7b1c3b28616a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:21.329718 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9481ccdd-2b4d-49aa-a266-7b1c3b28616a None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:21.329998 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9481ccdd-2b4d-49aa-a266-7b1c3b28616a None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:21.330268 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9481ccdd-2b4d-49aa-a266-7b1c3b28616a None admin] PATH_INFO: `/v3/projects/c2a0c48ef166451b912daa9e82ddcd09` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:21.331492 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9481ccdd-2b4d-49aa-a266-7b1c3b28616a None admin] RBAC: Authorizing `identity:get_project(project_id=c2a0c48ef166451b912daa9e82ddcd09)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:21.333277 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9481ccdd-2b4d-49aa-a266-7b1c3b28616a None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:21.336155 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 128/251] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 24 06:21:21 2022] GET /identity/v3/projects/c2a0c48ef166451b912daa9e82ddcd09 => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:21.338087 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9481ccdd-2b4d-49aa-a266-7b1c3b28616a None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:21.405848 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 124/252] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:21:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:21:21.406529 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa24c920-6d3c-47a1-9d40-6fe4b7da8885 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:21.428311 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:21.490524 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:21.518129 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9481ccdd-2b4d-49aa-a266-7b1c3b28616a None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:21.523034 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa24c920-6d3c-47a1-9d40-6fe4b7da8885 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:21.561723 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc24ab29-56d1-4edd-bf38-eae4e29fc1df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:21.562452 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc24ab29-56d1-4edd-bf38-eae4e29fc1df None admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:21.562720 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc24ab29-56d1-4edd-bf38-eae4e29fc1df None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:21.563068 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc24ab29-56d1-4edd-bf38-eae4e29fc1df None admin] PATH_INFO: `/v3/projects/c2a0c48ef166451b912daa9e82ddcd09/users/a15565acd41d402aaabba3fc11ec9e7a/roles/e1f9b167c51c4a388cbc9594db8a465d` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:21.565800 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc24ab29-56d1-4edd-bf38-eae4e29fc1df None admin] RBAC: Authorizing `identity:create_grant(project_id=c2a0c48ef166451b912daa9e82ddcd09, user_id=a15565acd41d402aaabba3fc11ec9e7a, role_id=e1f9b167c51c4a388cbc9594db8a465d)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:21.567311 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc24ab29-56d1-4edd-bf38-eae4e29fc1df None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:21.585411 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d359070a-1ca7-4239-be31-52803f0132db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:21.586058 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d359070a-1ca7-4239-be31-52803f0132db None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:21.586420 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d359070a-1ca7-4239-be31-52803f0132db None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:21.586699 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d359070a-1ca7-4239-be31-52803f0132db None admin] PATH_INFO: `/v3/projects/c2a0c48ef166451b912daa9e82ddcd09` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:21.587894 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d359070a-1ca7-4239-be31-52803f0132db None admin] RBAC: Authorizing `identity:get_project(project_id=c2a0c48ef166451b912daa9e82ddcd09)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:21.589529 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d359070a-1ca7-4239-be31-52803f0132db None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:21.591286 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 129/253] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 24 06:21:21 2022] GET /identity/v3/projects/c2a0c48ef166451b912daa9e82ddcd09 => generated 302 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:21.597686 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d359070a-1ca7-4239-be31-52803f0132db None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:21.649542 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 125/254] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Aug 24 06:21:21 2022] PUT /identity/v3/projects/c2a0c48ef166451b912daa9e82ddcd09/users/a15565acd41d402aaabba3fc11ec9e7a/roles/e1f9b167c51c4a388cbc9594db8a465d => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:21:21.726360 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d359070a-1ca7-4239-be31-52803f0132db None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:21.768382 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de456fe3-9f91-4bf0-a613-9a772bc8cf3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:21.768995 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de456fe3-9f91-4bf0-a613-9a772bc8cf3e None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:21.769299 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de456fe3-9f91-4bf0-a613-9a772bc8cf3e None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:21.769570 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de456fe3-9f91-4bf0-a613-9a772bc8cf3e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:21.770299 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de456fe3-9f91-4bf0-a613-9a772bc8cf3e None admin] RBAC: Adding query filter params (role.id=fcfa05f1d2b1474bb4fc20fc988cacd9, scope.project.id=c2a0c48ef166451b912daa9e82ddcd09, user.id=360d2cdeb8fb48b89b7ac189977b0daa) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:21.770602 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de456fe3-9f91-4bf0-a613-9a772bc8cf3e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:21.772204 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de456fe3-9f91-4bf0-a613-9a772bc8cf3e None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:21.779230 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 130/255] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 24 06:21:21 2022] GET /identity/v3/role_assignments?user.id=360d2cdeb8fb48b89b7ac189977b0daa&scope.project.id=c2a0c48ef166451b912daa9e82ddcd09&role.id=fcfa05f1d2b1474bb4fc20fc988cacd9 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:21:21.823924 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc24ab29-56d1-4edd-bf38-eae4e29fc1df None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:21.867812 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d14432-53c7-4c5f-9336-9190c16168e9 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:21.868453 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d14432-53c7-4c5f-9336-9190c16168e9 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:21.868929 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d14432-53c7-4c5f-9336-9190c16168e9 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:21.871090 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 131/256] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:21:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:21:21.878788 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7138ff-2d8e-46a9-a88d-01e45f95e409 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:21.879416 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7138ff-2d8e-46a9-a88d-01e45f95e409 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:21.879938 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7138ff-2d8e-46a9-a88d-01e45f95e409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:21.913332 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-6b7138ff-2d8e-46a9-a88d-01e45f95e409 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:21:21.914911 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-6b7138ff-2d8e-46a9-a88d-01e45f95e409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:21:21.933593 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:21.948645 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:21.990441 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc24ab29-56d1-4edd-bf38-eae4e29fc1df None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:22.043794 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 132/257] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:21:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:21:22.053051 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7419830-e548-4bdf-ab12-6db6a801f4c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:22.054038 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7419830-e548-4bdf-ab12-6db6a801f4c4 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:22.054516 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7419830-e548-4bdf-ab12-6db6a801f4c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:22.055008 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7419830-e548-4bdf-ab12-6db6a801f4c4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:22.069314 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7419830-e548-4bdf-ab12-6db6a801f4c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:21:22.074581 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-d7419830-e548-4bdf-ab12-6db6a801f4c4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:21:22.074581 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:21:22.074581 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:21:22.074581 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:21:22.074581 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:21:22.074581 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:21:22.074581 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:21:22.074581 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:21:22.074581 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:21:22.074581 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:21:22.074581 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:21:22.074581 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:21:22.074581 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:21:22.074581 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:21:22.074581 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:21:22.074581 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:21:22.075436 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:21:22.075436 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:21:22.075436 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:21:22.075436 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:21:22.075436 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:21:22.075436 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:21:22.075436 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:21:22.075436 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:21:22.075436 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:21:22.075436 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:21:22.075436 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:21:22.075436 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:21:22.075436 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:21:22.075436 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:21:22.075436 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:21:22.075436 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:21:22.076211 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:21:22.076211 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:21:22.076211 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:21:22.076211 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:21:22.076211 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:21:22.076211 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:21:22.076211 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:21:22.076211 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:21:22.076211 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:21:22.076211 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:21:22.076211 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:21:22.076211 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:21:22.084706 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 126/258] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Aug 24 06:21:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:21:22.086390 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7138ff-2d8e-46a9-a88d-01e45f95e409 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:22.208577 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7138ff-2d8e-46a9-a88d-01e45f95e409 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:22.247358 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4959709d-b9a5-484f-b01a-d2431d33ea67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:22.248377 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4959709d-b9a5-484f-b01a-d2431d33ea67 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:22.249546 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4959709d-b9a5-484f-b01a-d2431d33ea67 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:22.250044 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4959709d-b9a5-484f-b01a-d2431d33ea67 None admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:22.250906 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4959709d-b9a5-484f-b01a-d2431d33ea67 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:22.251404 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4959709d-b9a5-484f-b01a-d2431d33ea67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:22.252975 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4959709d-b9a5-484f-b01a-d2431d33ea67 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:22.264566 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 133/259] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 24 06:21:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:22.269451 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7419830-e548-4bdf-ab12-6db6a801f4c4 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:22.369501 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4959709d-b9a5-484f-b01a-d2431d33ea67 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:22.388674 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:22.435427 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7419830-e548-4bdf-ab12-6db6a801f4c4 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:22.494860 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:22.495771 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4959709d-b9a5-484f-b01a-d2431d33ea67 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:22.506148 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e364e7fe-db8b-4ea2-9c4c-e0a9b602d1ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:22.506891 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e364e7fe-db8b-4ea2-9c4c-e0a9b602d1ee None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:22.507180 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e364e7fe-db8b-4ea2-9c4c-e0a9b602d1ee None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:22.507459 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e364e7fe-db8b-4ea2-9c4c-e0a9b602d1ee None admin] PATH_INFO: `/v3/users/94a0cbc0f0d2429781903e722bf6eb53` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:22.508726 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e364e7fe-db8b-4ea2-9c4c-e0a9b602d1ee None admin] RBAC: Authorizing `identity:get_user(user_id=94a0cbc0f0d2429781903e722bf6eb53)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:22.510310 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e364e7fe-db8b-4ea2-9c4c-e0a9b602d1ee None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:22.512265 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 127/260] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 24 06:21:22 2022] GET /identity/v3/users/94a0cbc0f0d2429781903e722bf6eb53 => generated 283 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:22.519432 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e364e7fe-db8b-4ea2-9c4c-e0a9b602d1ee None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:22.543655 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea408ea-83d3-4588-8157-750a5dad62ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:22.544304 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea408ea-83d3-4588-8157-750a5dad62ae None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:22.544587 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea408ea-83d3-4588-8157-750a5dad62ae None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:22.544857 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea408ea-83d3-4588-8157-750a5dad62ae None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:22.558026 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea408ea-83d3-4588-8157-750a5dad62ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:21:22.563780 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-9ea408ea-83d3-4588-8157-750a5dad62ae None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 06:21:22.563780 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:21:22.563780 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:21:22.563780 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:21:22.563780 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:21:22.563780 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:21:22.563780 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:21:22.563780 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:21:22.563780 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:21:22.563780 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:21:22.563780 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:21:22.563780 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:21:22.563780 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:21:22.563780 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:21:22.563780 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:21:22.563780 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:21:22.564687 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:21:22.564687 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:21:22.564687 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:21:22.564687 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:21:22.564687 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:21:22.564687 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:21:22.564687 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:21:22.564687 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:21:22.564687 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:21:22.564687 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:21:22.564687 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:21:22.564687 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:21:22.564687 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:21:22.564687 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:21:22.564687 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:21:22.564687 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:21:22.565955 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:21:22.565955 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:21:22.565955 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:21:22.565955 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:21:22.565955 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:21:22.565955 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:21:22.565955 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:21:22.565955 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:21:22.565955 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:21:22.565955 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:21:22.565955 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 06:21:22.565955 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:21:22.565955 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 134/261] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Aug 24 06:21:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:21:22.571380 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea408ea-83d3-4588-8157-750a5dad62ae None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:22.679410 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:22.727610 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea408ea-83d3-4588-8157-750a5dad62ae None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:22.747258 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e364e7fe-db8b-4ea2-9c4c-e0a9b602d1ee None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:22.781875 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6a58da-021b-4c22-a469-950c9ecb3c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:22.783478 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6a58da-021b-4c22-a469-950c9ecb3c12 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:22.784042 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6a58da-021b-4c22-a469-950c9ecb3c12 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:22.784614 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6a58da-021b-4c22-a469-950c9ecb3c12 None admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:22.785541 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6a58da-021b-4c22-a469-950c9ecb3c12 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:22.786018 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6a58da-021b-4c22-a469-950c9ecb3c12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:22.788086 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6a58da-021b-4c22-a469-950c9ecb3c12 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:22.798375 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 135/262] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 24 06:21:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:21:22.806136 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6a58da-021b-4c22-a469-950c9ecb3c12 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:22.830687 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c3f509-ad38-459c-96bd-3b331eea60d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:22.831415 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c3f509-ad38-459c-96bd-3b331eea60d5 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:22.831766 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c3f509-ad38-459c-96bd-3b331eea60d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:22.832109 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c3f509-ad38-459c-96bd-3b331eea60d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:23.002296 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6a58da-021b-4c22-a469-950c9ecb3c12 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:23.031501 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:23.033439 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c3f509-ad38-459c-96bd-3b331eea60d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:23.034951 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c3f509-ad38-459c-96bd-3b331eea60d5 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:23.053842 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:23.063961 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78a4db0-779d-43bd-a466-d15ab5719cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:23.064619 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a4db0-779d-43bd-a466-d15ab5719cf6 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:23.064900 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a4db0-779d-43bd-a466-d15ab5719cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:23.065193 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a4db0-779d-43bd-a466-d15ab5719cf6 None admin] PATH_INFO: `/v3/users/94a0cbc0f0d2429781903e722bf6eb53` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:23.069966 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78a4db0-779d-43bd-a466-d15ab5719cf6 None admin] RBAC: Authorizing `identity:get_user(user_id=94a0cbc0f0d2429781903e722bf6eb53)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:23.071493 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78a4db0-779d-43bd-a466-d15ab5719cf6 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:23.073522 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 136/263] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 24 06:21:22 2022] GET /identity/v3/users/94a0cbc0f0d2429781903e722bf6eb53 => generated 283 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:23.079101 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78a4db0-779d-43bd-a466-d15ab5719cf6 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:23.243402 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78a4db0-779d-43bd-a466-d15ab5719cf6 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:23.274850 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 128/264] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 24 06:21:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 756 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 06:21:23.281140 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c3f509-ad38-459c-96bd-3b331eea60d5 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:23.299801 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a385ee18-3c42-410c-9e4f-b594da52470f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:23.300426 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a385ee18-3c42-410c-9e4f-b594da52470f None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:23.300705 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a385ee18-3c42-410c-9e4f-b594da52470f None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:23.300975 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a385ee18-3c42-410c-9e4f-b594da52470f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:23.416315 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:23.418864 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a385ee18-3c42-410c-9e4f-b594da52470f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:23.420931 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a385ee18-3c42-410c-9e4f-b594da52470f None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:23.489479 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c3f509-ad38-459c-96bd-3b331eea60d5 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:23.505552 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:23.550990 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2c4e58-fe66-444f-be23-5101efa271fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:23.551566 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2c4e58-fe66-444f-be23-5101efa271fb None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:23.551851 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2c4e58-fe66-444f-be23-5101efa271fb None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:23.552130 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2c4e58-fe66-444f-be23-5101efa271fb None admin] PATH_INFO: `/v3/projects/c2a0c48ef166451b912daa9e82ddcd09` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:23.560112 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 137/265] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 24 06:21:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 06:21:23.561928 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2c4e58-fe66-444f-be23-5101efa271fb None admin] RBAC: Authorizing `identity:get_project(project_id=c2a0c48ef166451b912daa9e82ddcd09)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:23.563711 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2c4e58-fe66-444f-be23-5101efa271fb None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:23.566316 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a385ee18-3c42-410c-9e4f-b594da52470f None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:23.567158 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 129/266] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 24 06:21:23 2022] GET /identity/v3/projects/c2a0c48ef166451b912daa9e82ddcd09 => generated 302 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:23.572058 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2c4e58-fe66-444f-be23-5101efa271fb None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:23.741209 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2c4e58-fe66-444f-be23-5101efa271fb None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:23.787818 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b460db3a-2b36-4ef6-8fdc-fd850fe4144c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:23.788447 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b460db3a-2b36-4ef6-8fdc-fd850fe4144c None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:23.788726 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b460db3a-2b36-4ef6-8fdc-fd850fe4144c None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:23.788998 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b460db3a-2b36-4ef6-8fdc-fd850fe4144c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:23.789770 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b460db3a-2b36-4ef6-8fdc-fd850fe4144c None admin] RBAC: Adding query filter params (role.id=486144ad1bcd417e86acdbc1d14327bf, scope.project.id=c2a0c48ef166451b912daa9e82ddcd09, user.id=94a0cbc0f0d2429781903e722bf6eb53) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:23.790074 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b460db3a-2b36-4ef6-8fdc-fd850fe4144c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:23.791528 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b460db3a-2b36-4ef6-8fdc-fd850fe4144c None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:23.803588 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a385ee18-3c42-410c-9e4f-b594da52470f None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:23.809230 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:23.809397 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 130/267] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 24 06:21:23 2022] GET /identity/v3/role_assignments?user.id=94a0cbc0f0d2429781903e722bf6eb53&scope.project.id=c2a0c48ef166451b912daa9e82ddcd09&role.id=486144ad1bcd417e86acdbc1d14327bf => generated 608 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:23.873921 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9aa82b-1482-4253-bb9e-bec911e3170f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:23.875033 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9aa82b-1482-4253-bb9e-bec911e3170f None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:23.875502 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9aa82b-1482-4253-bb9e-bec911e3170f None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:23.875955 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9aa82b-1482-4253-bb9e-bec911e3170f None admin] PATH_INFO: `/v3/projects/c2a0c48ef166451b912daa9e82ddcd09` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:23.882189 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9aa82b-1482-4253-bb9e-bec911e3170f None admin] RBAC: Authorizing `identity:get_project(project_id=c2a0c48ef166451b912daa9e82ddcd09)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:23.884201 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9aa82b-1482-4253-bb9e-bec911e3170f None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:23.886217 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 138/268] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 24 06:21:23 2022] GET /identity/v3/projects/c2a0c48ef166451b912daa9e82ddcd09 => generated 302 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:21:23.892591 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b460db3a-2b36-4ef6-8fdc-fd850fe4144c None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:23.947716 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:24.042396 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b460db3a-2b36-4ef6-8fdc-fd850fe4144c None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:24.081936 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd118242-c7d1-4848-9dd4-08f97def434b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:24.082483 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd118242-c7d1-4848-9dd4-08f97def434b None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:24.082803 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd118242-c7d1-4848-9dd4-08f97def434b None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:24.083040 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd118242-c7d1-4848-9dd4-08f97def434b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:24.083785 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd118242-c7d1-4848-9dd4-08f97def434b None admin] RBAC: Adding query filter params (role.id=b71c87e12dc1452ea3c99a7bf28585ca, scope.project.id=c2a0c48ef166451b912daa9e82ddcd09, user.id=94a0cbc0f0d2429781903e722bf6eb53) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:24.084095 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd118242-c7d1-4848-9dd4-08f97def434b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:24.085573 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd118242-c7d1-4848-9dd4-08f97def434b None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:24.092823 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 131/269] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 24 06:21:23 2022] GET /identity/v3/role_assignments?user.id=94a0cbc0f0d2429781903e722bf6eb53&scope.project.id=c2a0c48ef166451b912daa9e82ddcd09&role.id=b71c87e12dc1452ea3c99a7bf28585ca => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:24.199049 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ed4b1b-91a2-41e5-8186-5c5340ecc468 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:24.199698 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ed4b1b-91a2-41e5-8186-5c5340ecc468 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:24.200176 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ed4b1b-91a2-41e5-8186-5c5340ecc468 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:24.202849 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 139/270] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:21:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:21:24.208947 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5f4e0b-232e-42f5-9a76-5cff32f1c61f None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:24.209269 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5f4e0b-232e-42f5-9a76-5cff32f1c61f None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:24.209536 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5f4e0b-232e-42f5-9a76-5cff32f1c61f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:24.229957 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-1c5f4e0b-232e-42f5-9a76-5cff32f1c61f None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:21:24.231492 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-1c5f4e0b-232e-42f5-9a76-5cff32f1c61f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:21:24.312384 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ae81fb-b8a9-4d7e-be7c-98737044940e None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:24.313014 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ae81fb-b8a9-4d7e-be7c-98737044940e None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:24.313509 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ae81fb-b8a9-4d7e-be7c-98737044940e None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:24.315354 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 140/272] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:21:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:21:24.315462 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 132/272] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:21:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 06:21:24.320620 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36d4b95-a939-4998-beca-4f4d5640514f None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:24.320847 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36d4b95-a939-4998-beca-4f4d5640514f None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:24.321122 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36d4b95-a939-4998-beca-4f4d5640514f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:24.340374 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-d36d4b95-a939-4998-beca-4f4d5640514f None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:21:24.341537 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-d36d4b95-a939-4998-beca-4f4d5640514f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:21:24.370291 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:24.423971 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 133/273] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:21:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 06:21:24.629353 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ae81fb-b8a9-4d7e-be7c-98737044940e None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:24.761939 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ae81fb-b8a9-4d7e-be7c-98737044940e None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:24.802799 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493dc0a2-034a-42b0-8da9-ddd12332c3c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:24.803452 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493dc0a2-034a-42b0-8da9-ddd12332c3c0 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:24.803929 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493dc0a2-034a-42b0-8da9-ddd12332c3c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:24.805141 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36d4b95-a939-4998-beca-4f4d5640514f None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:24.806422 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493dc0a2-034a-42b0-8da9-ddd12332c3c0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:24.808446 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493dc0a2-034a-42b0-8da9-ddd12332c3c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:24.811820 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493dc0a2-034a-42b0-8da9-ddd12332c3c0 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:24.814188 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 141/274] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:21:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:24.819409 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493dc0a2-034a-42b0-8da9-ddd12332c3c0 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:24.861277 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:24.938778 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493dc0a2-034a-42b0-8da9-ddd12332c3c0 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:24.958774 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36d4b95-a939-4998-beca-4f4d5640514f None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:24.975299 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b06f7b4-a054-4dfe-b0fc-00beeb58db84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:24.976117 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b06f7b4-a054-4dfe-b0fc-00beeb58db84 None admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:24.976392 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b06f7b4-a054-4dfe-b0fc-00beeb58db84 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:24.976665 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b06f7b4-a054-4dfe-b0fc-00beeb58db84 None admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:24.977346 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b06f7b4-a054-4dfe-b0fc-00beeb58db84 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:24.978881 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b06f7b4-a054-4dfe-b0fc-00beeb58db84 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:24.994784 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacfe3be-ce21-44de-ad48-eed3f01a98af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:24.995427 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacfe3be-ce21-44de-ad48-eed3f01a98af None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:24.995701 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacfe3be-ce21-44de-ad48-eed3f01a98af None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:24.995971 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacfe3be-ce21-44de-ad48-eed3f01a98af None admin] PATH_INFO: `/v3/roles/member` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:25.007226 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacfe3be-ce21-44de-ad48-eed3f01a98af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:21:25.012091 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-aacfe3be-ce21-44de-ad48-eed3f01a98af None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 06:21:25.012091 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:21:25.012091 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:21:25.012091 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:21:25.012091 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:21:25.012091 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:21:25.012091 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:21:25.012091 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:21:25.012091 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:21:25.012091 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:21:25.012091 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:21:25.012091 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:21:25.012091 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:21:25.012091 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:21:25.012091 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:21:25.012091 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:21:25.013292 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:21:25.013292 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:21:25.013292 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:21:25.013292 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:21:25.013292 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:21:25.013292 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:21:25.013292 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:21:25.013292 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:21:25.013292 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:21:25.013292 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:21:25.013292 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:21:25.013292 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:21:25.013292 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:21:25.013292 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:21:25.013292 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:21:25.013292 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:21:25.014751 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:21:25.014751 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:21:25.014751 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:21:25.014751 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:21:25.014751 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:21:25.014751 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:21:25.014751 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:21:25.014751 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:21:25.014751 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:21:25.014751 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:21:25.014751 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 06:21:25.014751 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:21:25.014751 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 134/275] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 24 06:21:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:21:25.015633 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867ee935-5b49-4df6-b440-a914336aad90 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:25.015633 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867ee935-5b49-4df6-b440-a914336aad90 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:25.015785 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867ee935-5b49-4df6-b440-a914336aad90 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:25.017604 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 135/276] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:21:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:21:25.021326 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d7502-caaf-449c-8e7d-a6c520d9deaa None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:25.021326 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d7502-caaf-449c-8e7d-a6c520d9deaa None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:25.021326 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d7502-caaf-449c-8e7d-a6c520d9deaa None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:25.021326 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 136/277] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:21:25 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:21:25.021326 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12d7502-caaf-449c-8e7d-a6c520d9deaa None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:25.040142 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:25.086171 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 142/278] 10.222.0.47 () {60 vars in 1132 bytes} [Wed Aug 24 06:21:24 2022] POST /identity/v3/users => generated 293 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:21:25.087503 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc641a30-14eb-49ae-a072-b2d30dc6a2ed None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:25.087967 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc641a30-14eb-49ae-a072-b2d30dc6a2ed None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:25.088245 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc641a30-14eb-49ae-a072-b2d30dc6a2ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:25.113348 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-cc641a30-14eb-49ae-a072-b2d30dc6a2ed None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:21:25.113348 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-cc641a30-14eb-49ae-a072-b2d30dc6a2ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:21:25.141297 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12d7502-caaf-449c-8e7d-a6c520d9deaa None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:25.186549 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82227795-6dfd-47e2-acca-426a2390bb19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:25.187296 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82227795-6dfd-47e2-acca-426a2390bb19 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:25.187632 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82227795-6dfd-47e2-acca-426a2390bb19 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:25.188083 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82227795-6dfd-47e2-acca-426a2390bb19 None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:25.188769 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82227795-6dfd-47e2-acca-426a2390bb19 None admin] RBAC: Adding query filter params (name=member) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:25.189164 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82227795-6dfd-47e2-acca-426a2390bb19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:25.190506 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82227795-6dfd-47e2-acca-426a2390bb19 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:25.207144 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 137/279] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Aug 24 06:21:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:25.207376 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 143/280] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:21:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:21:25.207485 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c2b445-bbfd-4128-a32d-aff010279f85 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:25.207485 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c2b445-bbfd-4128-a32d-aff010279f85 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:25.207485 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c2b445-bbfd-4128-a32d-aff010279f85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:25.209539 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc641a30-14eb-49ae-a072-b2d30dc6a2ed None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:25.230377 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-b6c2b445-bbfd-4128-a32d-aff010279f85 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:21:25.231599 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-b6c2b445-bbfd-4128-a32d-aff010279f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:21:25.242181 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:25.332444 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 138/281] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:21:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:21:25.355660 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc641a30-14eb-49ae-a072-b2d30dc6a2ed None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:25.397988 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c532be2b-621f-44ce-8f1a-29e154156ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:25.398645 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c532be2b-621f-44ce-8f1a-29e154156ddd None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:25.398924 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c532be2b-621f-44ce-8f1a-29e154156ddd None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:25.399199 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c532be2b-621f-44ce-8f1a-29e154156ddd None admin] PATH_INFO: `/v3/users/762c5c56211f40df9a8ba87468d39c05` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:25.416891 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c532be2b-621f-44ce-8f1a-29e154156ddd None admin] RBAC: Authorizing `identity:get_user(user_id=762c5c56211f40df9a8ba87468d39c05)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:25.418398 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c532be2b-621f-44ce-8f1a-29e154156ddd None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:25.420741 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 144/282] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 24 06:21:25 2022] GET /identity/v3/users/762c5c56211f40df9a8ba87468d39c05 => generated 293 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:21:25.427173 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c2b445-bbfd-4128-a32d-aff010279f85 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:25.445621 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:25.478883 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c532be2b-621f-44ce-8f1a-29e154156ddd None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:25.556691 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c2b445-bbfd-4128-a32d-aff010279f85 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:25.589279 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:25.606093 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a608a8ca-cf46-4d59-95b9-b63731640a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:25.606982 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a608a8ca-cf46-4d59-95b9-b63731640a02 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:25.607271 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a608a8ca-cf46-4d59-95b9-b63731640a02 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:25.607717 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a608a8ca-cf46-4d59-95b9-b63731640a02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:25.608458 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a608a8ca-cf46-4d59-95b9-b63731640a02 None admin] RBAC: Adding query filter params (role.id=e1f9b167c51c4a388cbc9594db8a465d, user.id=762c5c56211f40df9a8ba87468d39c05, scope.system=all) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:25.608928 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a608a8ca-cf46-4d59-95b9-b63731640a02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:25.611081 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a608a8ca-cf46-4d59-95b9-b63731640a02 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:25.618770 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 139/283] 10.222.0.47 () {56 vars in 1308 bytes} [Wed Aug 24 06:21:25 2022] GET /identity/v3/role_assignments?user.id=762c5c56211f40df9a8ba87468d39c05&scope.system=all&role.id=e1f9b167c51c4a388cbc9594db8a465d => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:25.624984 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c532be2b-621f-44ce-8f1a-29e154156ddd None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:25.636197 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a608a8ca-cf46-4d59-95b9-b63731640a02 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:25.689163 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c387f5-c6ab-460e-94b1-dabaaf19a534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:25.689976 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c387f5-c6ab-460e-94b1-dabaaf19a534 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:25.690143 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c387f5-c6ab-460e-94b1-dabaaf19a534 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:25.690386 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c387f5-c6ab-460e-94b1-dabaaf19a534 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:25.705374 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c387f5-c6ab-460e-94b1-dabaaf19a534 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:21:25.710611 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-38c387f5-c6ab-460e-94b1-dabaaf19a534 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 06:21:25.710611 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:21:25.710611 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:21:25.710611 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:21:25.710611 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:21:25.710611 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:21:25.710611 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:21:25.710611 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:21:25.710611 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:21:25.710611 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:21:25.710611 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:21:25.710611 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:21:25.710611 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:21:25.710611 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:21:25.710611 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:21:25.710611 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:21:25.711554 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:21:25.711554 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:21:25.711554 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:21:25.711554 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:21:25.711554 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:21:25.711554 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:21:25.711554 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:21:25.711554 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:21:25.711554 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:21:25.711554 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:21:25.711554 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:21:25.711554 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:21:25.711554 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:21:25.711554 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:21:25.711554 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:21:25.711554 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:21:25.712997 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:21:25.712997 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:21:25.712997 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:21:25.712997 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:21:25.712997 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:21:25.712997 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:21:25.712997 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:21:25.712997 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:21:25.712997 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:21:25.712997 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:21:25.712997 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 06:21:25.712997 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:21:25.712997 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 145/284] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 24 06:21:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:21:25.718401 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c387f5-c6ab-460e-94b1-dabaaf19a534 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:25.765504 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:25.808150 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a608a8ca-cf46-4d59-95b9-b63731640a02 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:25.846514 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8e2fba-4499-46f1-b00b-780f824cf175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:25.847413 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8e2fba-4499-46f1-b00b-780f824cf175 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:25.847692 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8e2fba-4499-46f1-b00b-780f824cf175 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:25.847963 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8e2fba-4499-46f1-b00b-780f824cf175 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:25.937096 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8e2fba-4499-46f1-b00b-780f824cf175 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:21:25.938428 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c387f5-c6ab-460e-94b1-dabaaf19a534 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:25.938553 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-5f8e2fba-4499-46f1-b00b-780f824cf175 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 06:21:25.938553 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:21:25.938553 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:21:25.938553 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:21:25.938553 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:21:25.938553 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:21:25.938553 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:21:25.938553 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:21:25.938553 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:21:25.938553 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:21:25.938553 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:21:25.938553 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:21:25.938553 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:21:25.938553 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:21:25.938553 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:21:25.938553 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:21:25.939368 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:21:25.939368 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:21:25.939368 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:21:25.939368 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:21:25.939368 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:21:25.939368 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:21:25.939368 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:21:25.939368 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:21:25.939368 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:21:25.939368 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:21:25.939368 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:21:25.939368 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:21:25.939368 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:21:25.939368 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:21:25.939368 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:21:25.939368 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:21:25.940203 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:21:25.940203 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:21:25.940203 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:21:25.940203 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:21:25.940203 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:21:25.940203 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:21:25.940203 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:21:25.940203 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:21:25.940203 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:21:25.940203 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:21:25.940203 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 06:21:25.940203 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:21:25.940203 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 140/285] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 24 06:21:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:21:25.940203 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8e2fba-4499-46f1-b00b-780f824cf175 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:25.940897 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2b6504-e527-4816-ac35-096862ab2635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:25.940897 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2b6504-e527-4816-ac35-096862ab2635 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:25.940897 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2b6504-e527-4816-ac35-096862ab2635 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:25.940897 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2b6504-e527-4816-ac35-096862ab2635 None admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:25.941138 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2b6504-e527-4816-ac35-096862ab2635 None admin] RBAC: Adding query filter params (name=member) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:25.941138 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2b6504-e527-4816-ac35-096862ab2635 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:25.941138 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2b6504-e527-4816-ac35-096862ab2635 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:25.941138 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 146/286] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Aug 24 06:21:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:21:25.941138 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2b6504-e527-4816-ac35-096862ab2635 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:25.944167 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:26.008506 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8e2fba-4499-46f1-b00b-780f824cf175 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:26.075117 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2b6504-e527-4816-ac35-096862ab2635 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:26.100252 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:26.102355 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd016c43-d18c-4a9f-b3ce-f742b26334f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:26.103208 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd016c43-d18c-4a9f-b3ce-f742b26334f3 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:26.103791 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd016c43-d18c-4a9f-b3ce-f742b26334f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:26.104251 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd016c43-d18c-4a9f-b3ce-f742b26334f3 None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:26.105203 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd016c43-d18c-4a9f-b3ce-f742b26334f3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:26.105698 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd016c43-d18c-4a9f-b3ce-f742b26334f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:26.107874 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd016c43-d18c-4a9f-b3ce-f742b26334f3 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:26.113400 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b91a7d-be3b-4337-bb47-ae9c3c743b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:26.117206 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b91a7d-be3b-4337-bb47-ae9c3c743b62 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:26.117206 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b91a7d-be3b-4337-bb47-ae9c3c743b62 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:26.117206 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b91a7d-be3b-4337-bb47-ae9c3c743b62 None admin] PATH_INFO: `/v3/users/a15565acd41d402aaabba3fc11ec9e7a` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:26.117206 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b91a7d-be3b-4337-bb47-ae9c3c743b62 None admin] RBAC: Authorizing `identity:get_user(user_id=a15565acd41d402aaabba3fc11ec9e7a)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:26.120370 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b91a7d-be3b-4337-bb47-ae9c3c743b62 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:26.120370 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 147/287] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 24 06:21:25 2022] GET /identity/v3/users/a15565acd41d402aaabba3fc11ec9e7a => generated 297 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:26.127436 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b91a7d-be3b-4337-bb47-ae9c3c743b62 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:26.129973 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 141/288] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Aug 24 06:21:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:26.136641 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd016c43-d18c-4a9f-b3ce-f742b26334f3 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:26.249551 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:26.309320 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd016c43-d18c-4a9f-b3ce-f742b26334f3 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:26.310611 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b91a7d-be3b-4337-bb47-ae9c3c743b62 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:26.351766 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474f3958-8c3b-4c7e-bc05-2fd87d64c008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:26.352415 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474f3958-8c3b-4c7e-bc05-2fd87d64c008 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:26.352691 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474f3958-8c3b-4c7e-bc05-2fd87d64c008 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:26.352962 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474f3958-8c3b-4c7e-bc05-2fd87d64c008 None admin] PATH_INFO: `/v3/users/360d2cdeb8fb48b89b7ac189977b0daa` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:26.355026 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474f3958-8c3b-4c7e-bc05-2fd87d64c008 None admin] RBAC: Authorizing `identity:get_user(user_id=360d2cdeb8fb48b89b7ac189977b0daa)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:26.356523 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474f3958-8c3b-4c7e-bc05-2fd87d64c008 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:26.358204 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 142/289] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 24 06:21:26 2022] GET /identity/v3/users/360d2cdeb8fb48b89b7ac189977b0daa => generated 297 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:26.368076 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474f3958-8c3b-4c7e-bc05-2fd87d64c008 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:26.400263 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9e8b25-16a8-4dfc-8eef-d581bc67228b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:26.400893 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9e8b25-16a8-4dfc-8eef-d581bc67228b None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:26.401175 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9e8b25-16a8-4dfc-8eef-d581bc67228b None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:26.401472 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9e8b25-16a8-4dfc-8eef-d581bc67228b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:26.491114 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9e8b25-16a8-4dfc-8eef-d581bc67228b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:26.492576 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9e8b25-16a8-4dfc-8eef-d581bc67228b None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:26.545572 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:26.567739 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:26.609600 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474f3958-8c3b-4c7e-bc05-2fd87d64c008 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:26.650990 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 148/290] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 24 06:21:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 06:21:26.657029 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9e8b25-16a8-4dfc-8eef-d581bc67228b None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:26.658225 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ff3082-3258-45c8-8d7f-7574b95ab53c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:26.658885 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ff3082-3258-45c8-8d7f-7574b95ab53c None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:26.659407 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ff3082-3258-45c8-8d7f-7574b95ab53c None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:26.659681 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ff3082-3258-45c8-8d7f-7574b95ab53c None admin] PATH_INFO: `/v3/projects/c2a0c48ef166451b912daa9e82ddcd09` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:26.662251 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ff3082-3258-45c8-8d7f-7574b95ab53c None admin] RBAC: Authorizing `identity:get_project(project_id=c2a0c48ef166451b912daa9e82ddcd09)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:26.663870 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ff3082-3258-45c8-8d7f-7574b95ab53c None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:26.666076 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 143/291] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 24 06:21:26 2022] GET /identity/v3/projects/c2a0c48ef166451b912daa9e82ddcd09 => generated 302 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:26.675490 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ff3082-3258-45c8-8d7f-7574b95ab53c None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:26.806372 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ff3082-3258-45c8-8d7f-7574b95ab53c None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:26.841004 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:26.846274 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdef8b3-56a1-4ffe-858a-9c23b4e10f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:26.846953 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdef8b3-56a1-4ffe-858a-9c23b4e10f06 None admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:26.847231 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdef8b3-56a1-4ffe-858a-9c23b4e10f06 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:26.847508 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdef8b3-56a1-4ffe-858a-9c23b4e10f06 None admin] PATH_INFO: `/v3/projects/c2a0c48ef166451b912daa9e82ddcd09/users/360d2cdeb8fb48b89b7ac189977b0daa/roles/fcfa05f1d2b1474bb4fc20fc988cacd9` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:26.849701 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdef8b3-56a1-4ffe-858a-9c23b4e10f06 None admin] RBAC: Authorizing `identity:create_grant(project_id=c2a0c48ef166451b912daa9e82ddcd09, user_id=360d2cdeb8fb48b89b7ac189977b0daa, role_id=fcfa05f1d2b1474bb4fc20fc988cacd9)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:26.851326 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdef8b3-56a1-4ffe-858a-9c23b4e10f06 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:26.886880 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9e8b25-16a8-4dfc-8eef-d581bc67228b None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:26.927598 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37337b3-7a3a-444c-ba11-c1c77f684fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:26.928352 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37337b3-7a3a-444c-ba11-c1c77f684fb1 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:26.928631 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37337b3-7a3a-444c-ba11-c1c77f684fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:26.928903 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37337b3-7a3a-444c-ba11-c1c77f684fb1 None admin] PATH_INFO: `/v3/projects/c2a0c48ef166451b912daa9e82ddcd09` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:26.930191 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37337b3-7a3a-444c-ba11-c1c77f684fb1 None admin] RBAC: Authorizing `identity:get_project(project_id=c2a0c48ef166451b912daa9e82ddcd09)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:26.931724 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37337b3-7a3a-444c-ba11-c1c77f684fb1 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:26.933870 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 149/292] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 24 06:21:26 2022] GET /identity/v3/projects/c2a0c48ef166451b912daa9e82ddcd09 => generated 302 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:26.942920 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37337b3-7a3a-444c-ba11-c1c77f684fb1 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:26.993395 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 144/293] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Aug 24 06:21:26 2022] PUT /identity/v3/projects/c2a0c48ef166451b912daa9e82ddcd09/users/360d2cdeb8fb48b89b7ac189977b0daa/roles/fcfa05f1d2b1474bb4fc20fc988cacd9 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:21:26.993591 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:27.065669 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37337b3-7a3a-444c-ba11-c1c77f684fb1 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:27.104132 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b82c193-1149-440c-a4de-94dd8c39d3bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:27.104670 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b82c193-1149-440c-a4de-94dd8c39d3bd None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:27.104955 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b82c193-1149-440c-a4de-94dd8c39d3bd None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:27.105259 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b82c193-1149-440c-a4de-94dd8c39d3bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:27.106103 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b82c193-1149-440c-a4de-94dd8c39d3bd None admin] RBAC: Adding query filter params (role.id=e1f9b167c51c4a388cbc9594db8a465d, scope.project.id=c2a0c48ef166451b912daa9e82ddcd09, user.id=a15565acd41d402aaabba3fc11ec9e7a) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:27.106411 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b82c193-1149-440c-a4de-94dd8c39d3bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:27.107966 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b82c193-1149-440c-a4de-94dd8c39d3bd None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:27.115439 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 150/294] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 24 06:21:26 2022] GET /identity/v3/role_assignments?user.id=a15565acd41d402aaabba3fc11ec9e7a&scope.project.id=c2a0c48ef166451b912daa9e82ddcd09&role.id=e1f9b167c51c4a388cbc9594db8a465d => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:21:27.628655 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73474029-6d92-4174-874b-8bcc8aee866c None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:27.629291 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73474029-6d92-4174-874b-8bcc8aee866c None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:27.629291 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73474029-6d92-4174-874b-8bcc8aee866c None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:27.630570 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 145/295] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:21:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:21:27.640351 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e0f65f-8208-4cf6-8ed8-51397eafca2d None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:27.640351 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e0f65f-8208-4cf6-8ed8-51397eafca2d None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:27.640351 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e0f65f-8208-4cf6-8ed8-51397eafca2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:27.665264 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-20e0f65f-8208-4cf6-8ed8-51397eafca2d None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:21:27.666768 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-20e0f65f-8208-4cf6-8ed8-51397eafca2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:21:27.758109 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 151/296] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:21:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 06:21:27.812802 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fbb1fc-0973-4147-975f-2f4634ae48f0 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:27.813120 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fbb1fc-0973-4147-975f-2f4634ae48f0 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:27.813386 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fbb1fc-0973-4147-975f-2f4634ae48f0 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:27.814996 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 146/297] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:21:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:21:27.820552 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dfb84d-a7d4-4004-bb74-829d72e1fb64 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:27.821138 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dfb84d-a7d4-4004-bb74-829d72e1fb64 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:27.821701 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dfb84d-a7d4-4004-bb74-829d72e1fb64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:27.847831 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-00dfb84d-a7d4-4004-bb74-829d72e1fb64 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:21:27.849591 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-00dfb84d-a7d4-4004-bb74-829d72e1fb64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:21:27.887034 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:27.943792 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 152/298] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:21:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:21:28.067854 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fbb1fc-0973-4147-975f-2f4634ae48f0 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:28.119101 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a29f0f4-23ad-48df-856c-615318c68472 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:28.119380 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a29f0f4-23ad-48df-856c-615318c68472 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:28.119898 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a29f0f4-23ad-48df-856c-615318c68472 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:28.121506 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 153/299] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:21:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 06:21:28.127212 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e80f3d-6408-48af-9e57-8b4956f39857 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:28.127817 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e80f3d-6408-48af-9e57-8b4956f39857 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:28.128294 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e80f3d-6408-48af-9e57-8b4956f39857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:28.150188 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-74e80f3d-6408-48af-9e57-8b4956f39857 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:21:28.152831 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-74e80f3d-6408-48af-9e57-8b4956f39857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:21:28.204224 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fbb1fc-0973-4147-975f-2f4634ae48f0 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:28.243182 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65cb38a-10f2-4a9e-b2d2-ce678a24565c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:28.244219 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 154/300] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:21:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 06:21:28.245298 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65cb38a-10f2-4a9e-b2d2-ce678a24565c None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:28.245741 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65cb38a-10f2-4a9e-b2d2-ce678a24565c None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:28.246171 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65cb38a-10f2-4a9e-b2d2-ce678a24565c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:28.258977 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e80f3d-6408-48af-9e57-8b4956f39857 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:28.261899 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65cb38a-10f2-4a9e-b2d2-ce678a24565c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:21:28.269554 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-a65cb38a-10f2-4a9e-b2d2-ce678a24565c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 06:21:28.269554 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:21:28.269554 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:21:28.269554 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:21:28.269554 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:21:28.269554 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:21:28.269554 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:21:28.269554 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:21:28.269554 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:21:28.269554 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:21:28.269554 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:21:28.269554 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:21:28.269554 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:21:28.269554 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:21:28.269554 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:21:28.269554 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:21:28.270664 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:21:28.270664 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:21:28.270664 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:21:28.270664 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:21:28.270664 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:21:28.270664 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:21:28.270664 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:21:28.270664 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:21:28.270664 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:21:28.270664 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:21:28.270664 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:21:28.270664 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:21:28.270664 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:21:28.270664 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:21:28.270664 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:21:28.270664 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:21:28.271444 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:21:28.271444 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:21:28.271444 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:21:28.271444 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:21:28.271444 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:21:28.271444 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:21:28.271444 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:21:28.271444 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:21:28.271444 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:21:28.271444 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:21:28.271444 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 06:21:28.271444 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:21:28.272482 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 147/301] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 24 06:21:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:21:28.278555 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65cb38a-10f2-4a9e-b2d2-ce678a24565c None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:28.348212 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:28.402009 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:28.405972 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e80f3d-6408-48af-9e57-8b4956f39857 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:28.426205 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65cb38a-10f2-4a9e-b2d2-ce678a24565c None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:28.456651 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6ecb7c-59ce-432e-81af-54b9c82bff81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:28.457818 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6ecb7c-59ce-432e-81af-54b9c82bff81 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:28.458172 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6ecb7c-59ce-432e-81af-54b9c82bff81 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:28.458525 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6ecb7c-59ce-432e-81af-54b9c82bff81 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:28.459863 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6ecb7c-59ce-432e-81af-54b9c82bff81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:28.462345 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6ecb7c-59ce-432e-81af-54b9c82bff81 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:28.464611 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 155/302] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:21:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:21:28.470695 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6ecb7c-59ce-432e-81af-54b9c82bff81 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:28.479464 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65c361a-cf3e-470e-88df-b85f71f7bdc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:28.479464 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65c361a-cf3e-470e-88df-b85f71f7bdc2 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:28.479464 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65c361a-cf3e-470e-88df-b85f71f7bdc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:28.479464 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65c361a-cf3e-470e-88df-b85f71f7bdc2 None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:28.479877 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65c361a-cf3e-470e-88df-b85f71f7bdc2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:28.479877 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65c361a-cf3e-470e-88df-b85f71f7bdc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:28.482112 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65c361a-cf3e-470e-88df-b85f71f7bdc2 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:28.497283 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 148/303] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Aug 24 06:21:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:28.500605 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65c361a-cf3e-470e-88df-b85f71f7bdc2 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:28.621265 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6ecb7c-59ce-432e-81af-54b9c82bff81 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:28.647747 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65c361a-cf3e-470e-88df-b85f71f7bdc2 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:28.672998 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fdc36d-a355-4a58-9b5b-57b3050cad86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:28.673947 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fdc36d-a355-4a58-9b5b-57b3050cad86 None admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:28.674295 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fdc36d-a355-4a58-9b5b-57b3050cad86 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:28.674449 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:28.674586 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fdc36d-a355-4a58-9b5b-57b3050cad86 None admin] PATH_INFO: `/v3/groups` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:28.675323 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fdc36d-a355-4a58-9b5b-57b3050cad86 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:28.676807 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fdc36d-a355-4a58-9b5b-57b3050cad86 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:28.708795 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edcf198-cd8a-4eec-8fa1-a7051f05893f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:28.709690 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edcf198-cd8a-4eec-8fa1-a7051f05893f None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:28.710261 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edcf198-cd8a-4eec-8fa1-a7051f05893f None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:28.710536 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edcf198-cd8a-4eec-8fa1-a7051f05893f None admin] PATH_INFO: `/v3/users/b5ba3a8f3deb4f1384a5bd6a62dde7cf` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:28.730418 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edcf198-cd8a-4eec-8fa1-a7051f05893f None admin] RBAC: Authorizing `identity:get_user(user_id=b5ba3a8f3deb4f1384a5bd6a62dde7cf)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:28.733051 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edcf198-cd8a-4eec-8fa1-a7051f05893f None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:28.735007 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 149/304] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 24 06:21:28 2022] GET /identity/v3/users/b5ba3a8f3deb4f1384a5bd6a62dde7cf => generated 293 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:28.737006 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edcf198-cd8a-4eec-8fa1-a7051f05893f None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:28.764435 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 156/305] 10.222.0.47 () {60 vars in 1134 bytes} [Wed Aug 24 06:21:28 2022] POST /identity/v3/groups => generated 229 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:21:28.765466 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fdc36d-a355-4a58-9b5b-57b3050cad86 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:28.813424 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:28.903491 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edcf198-cd8a-4eec-8fa1-a7051f05893f None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:28.939994 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fdc36d-a355-4a58-9b5b-57b3050cad86 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:28.948054 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeee1f81-6ac7-4bb4-97ec-a0fa9ed872f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:28.949003 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeee1f81-6ac7-4bb4-97ec-a0fa9ed872f4 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:28.949513 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeee1f81-6ac7-4bb4-97ec-a0fa9ed872f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:28.949789 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeee1f81-6ac7-4bb4-97ec-a0fa9ed872f4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:28.962897 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeee1f81-6ac7-4bb4-97ec-a0fa9ed872f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:21:28.968561 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-aeee1f81-6ac7-4bb4-97ec-a0fa9ed872f4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 06:21:28.968561 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:21:28.968561 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:21:28.968561 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:21:28.968561 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:21:28.968561 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:21:28.968561 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:21:28.968561 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:21:28.968561 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:21:28.968561 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:21:28.968561 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:21:28.968561 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:21:28.968561 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:21:28.968561 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:21:28.968561 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:21:28.968561 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:21:28.970203 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:21:28.970203 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:21:28.970203 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:21:28.970203 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:21:28.970203 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:21:28.970203 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:21:28.970203 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:21:28.970203 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:21:28.970203 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:21:28.970203 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:21:28.970203 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:21:28.970203 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:21:28.970203 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:21:28.970203 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:21:28.970203 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:21:28.970203 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:21:28.971006 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:21:28.971006 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:21:28.971006 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:21:28.971006 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:21:28.971006 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:21:28.971006 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:21:28.971006 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:21:28.971006 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:21:28.971006 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:21:28.971006 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:21:28.971006 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 06:21:28.971006 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:21:28.971006 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 150/306] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 24 06:21:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:21:28.975838 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeee1f81-6ac7-4bb4-97ec-a0fa9ed872f4 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:28.997409 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5f4802-6e3a-487e-89e8-b7c28f196b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:28.998059 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5f4802-6e3a-487e-89e8-b7c28f196b51 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:28.998337 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5f4802-6e3a-487e-89e8-b7c28f196b51 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:28.998609 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5f4802-6e3a-487e-89e8-b7c28f196b51 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:28.999749 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5f4802-6e3a-487e-89e8-b7c28f196b51 None admin] RBAC: Adding query filter params (role.id=fcfa05f1d2b1474bb4fc20fc988cacd9, user.id=b5ba3a8f3deb4f1384a5bd6a62dde7cf, scope.system=all) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:28.999749 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5f4802-6e3a-487e-89e8-b7c28f196b51 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:29.001270 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5f4802-6e3a-487e-89e8-b7c28f196b51 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:29.008242 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 157/307] 10.222.0.47 () {56 vars in 1308 bytes} [Wed Aug 24 06:21:28 2022] GET /identity/v3/role_assignments?user.id=b5ba3a8f3deb4f1384a5bd6a62dde7cf&scope.system=all&role.id=fcfa05f1d2b1474bb4fc20fc988cacd9 => generated 226 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:29.019767 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:29.104140 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeee1f81-6ac7-4bb4-97ec-a0fa9ed872f4 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:29.142665 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2102edb-d1e4-45bf-87fd-2b10e12f7299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:29.143291 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2102edb-d1e4-45bf-87fd-2b10e12f7299 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:29.143566 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2102edb-d1e4-45bf-87fd-2b10e12f7299 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:29.143831 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2102edb-d1e4-45bf-87fd-2b10e12f7299 None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:29.144512 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2102edb-d1e4-45bf-87fd-2b10e12f7299 None admin] RBAC: Adding query filter params (name=member) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:29.144812 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2102edb-d1e4-45bf-87fd-2b10e12f7299 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:29.146312 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2102edb-d1e4-45bf-87fd-2b10e12f7299 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:29.156902 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 151/308] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Aug 24 06:21:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:21:29.163541 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5f4802-6e3a-487e-89e8-b7c28f196b51 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:29.261203 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:29.278673 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5f4802-6e3a-487e-89e8-b7c28f196b51 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:29.311652 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37631c2c-c083-473d-b9c9-14a87152828b None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:29.311943 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37631c2c-c083-473d-b9c9-14a87152828b None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:29.312247 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37631c2c-c083-473d-b9c9-14a87152828b None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:29.313894 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 152/309] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:21:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 06:21:29.319456 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68a1a97-51c1-4f11-a9ad-092c367fab3e None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:29.319746 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68a1a97-51c1-4f11-a9ad-092c367fab3e None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:29.319942 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2ba4bf-8cbf-4ca3-8c76-3f8443eaeece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:29.320091 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68a1a97-51c1-4f11-a9ad-092c367fab3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:29.320860 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2ba4bf-8cbf-4ca3-8c76-3f8443eaeece None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:29.321306 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2ba4bf-8cbf-4ca3-8c76-3f8443eaeece None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:29.321717 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2ba4bf-8cbf-4ca3-8c76-3f8443eaeece None admin] PATH_INFO: `/v3/users/762c5c56211f40df9a8ba87468d39c05` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:29.323015 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2ba4bf-8cbf-4ca3-8c76-3f8443eaeece None admin] RBAC: Authorizing `identity:get_user(user_id=762c5c56211f40df9a8ba87468d39c05)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:29.324704 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2ba4bf-8cbf-4ca3-8c76-3f8443eaeece None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:29.327615 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 158/310] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 24 06:21:29 2022] GET /identity/v3/users/762c5c56211f40df9a8ba87468d39c05 => generated 293 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:29.334366 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2ba4bf-8cbf-4ca3-8c76-3f8443eaeece None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:29.340647 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-e68a1a97-51c1-4f11-a9ad-092c367fab3e None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:21:29.342502 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-e68a1a97-51c1-4f11-a9ad-092c367fab3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:21:29.428424 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 153/311] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:21:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 06:21:29.475682 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2ba4bf-8cbf-4ca3-8c76-3f8443eaeece None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:29.517272 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2f0b79-ebe4-461a-bf38-0c6e6a5b58dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:29.518173 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2f0b79-ebe4-461a-bf38-0c6e6a5b58dd None admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:29.518603 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2f0b79-ebe4-461a-bf38-0c6e6a5b58dd None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:29.519097 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2f0b79-ebe4-461a-bf38-0c6e6a5b58dd None admin] PATH_INFO: `/v3/system/users/762c5c56211f40df9a8ba87468d39c05/roles/e1f9b167c51c4a388cbc9594db8a465d` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:29.520828 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2f0b79-ebe4-461a-bf38-0c6e6a5b58dd None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=762c5c56211f40df9a8ba87468d39c05, role_id=e1f9b167c51c4a388cbc9594db8a465d)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:29.522471 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2f0b79-ebe4-461a-bf38-0c6e6a5b58dd None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:29.612131 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 159/312] 10.222.0.47 () {58 vars in 1334 bytes} [Wed Aug 24 06:21:29 2022] PUT /identity/v3/system/users/762c5c56211f40df9a8ba87468d39c05/roles/e1f9b167c51c4a388cbc9594db8a465d => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:21:29.706868 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68a1a97-51c1-4f11-a9ad-092c367fab3e None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:29.735723 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:29.832047 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68a1a97-51c1-4f11-a9ad-092c367fab3e None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:29.873329 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd5a9f0-59ce-48e2-a098-24354aa74df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:29.874345 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd5a9f0-59ce-48e2-a098-24354aa74df8 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:29.874852 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd5a9f0-59ce-48e2-a098-24354aa74df8 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:29.875325 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd5a9f0-59ce-48e2-a098-24354aa74df8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:29.887943 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd5a9f0-59ce-48e2-a098-24354aa74df8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:21:29.893844 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-9cd5a9f0-59ce-48e2-a098-24354aa74df8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 06:21:29.893844 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:21:29.893844 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:21:29.893844 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:21:29.893844 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:21:29.893844 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:21:29.893844 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:21:29.893844 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:21:29.893844 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:21:29.893844 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:21:29.893844 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:21:29.893844 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:21:29.893844 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:21:29.893844 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:21:29.893844 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:21:29.893844 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:21:29.894778 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:21:29.894778 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:21:29.894778 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:21:29.894778 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:21:29.894778 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:21:29.894778 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:21:29.894778 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:21:29.894778 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:21:29.894778 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:21:29.894778 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:21:29.894778 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:21:29.894778 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:21:29.894778 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:21:29.894778 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:21:29.894778 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:21:29.894778 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:21:29.895574 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:21:29.895574 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:21:29.895574 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:21:29.895574 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:21:29.895574 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:21:29.895574 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:21:29.895574 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:21:29.895574 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:21:29.895574 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:21:29.895574 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:21:29.895574 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 06:21:29.895574 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:21:29.898254 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 154/313] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 24 06:21:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:21:29.910334 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2f0b79-ebe4-461a-bf38-0c6e6a5b58dd None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:29.930927 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:30.030840 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2f0b79-ebe4-461a-bf38-0c6e6a5b58dd None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:30.067458 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa05b79-ac15-4f3f-a0ea-ee2c3c42c524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:30.068081 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa05b79-ac15-4f3f-a0ea-ee2c3c42c524 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:30.068363 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa05b79-ac15-4f3f-a0ea-ee2c3c42c524 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:30.068639 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa05b79-ac15-4f3f-a0ea-ee2c3c42c524 None admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:30.069400 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa05b79-ac15-4f3f-a0ea-ee2c3c42c524 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:30.069709 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa05b79-ac15-4f3f-a0ea-ee2c3c42c524 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:30.071087 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa05b79-ac15-4f3f-a0ea-ee2c3c42c524 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:30.080821 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 160/314] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Aug 24 06:21:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:30.086602 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd5a9f0-59ce-48e2-a098-24354aa74df8 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:30.184441 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:30.208110 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd5a9f0-59ce-48e2-a098-24354aa74df8 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:30.243565 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa797c77-8e09-49ae-ba59-d41da90952ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:30.244232 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa797c77-8e09-49ae-ba59-d41da90952ca None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:30.244512 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa797c77-8e09-49ae-ba59-d41da90952ca None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:30.244783 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa797c77-8e09-49ae-ba59-d41da90952ca None admin] PATH_INFO: `/v3/users/360d2cdeb8fb48b89b7ac189977b0daa` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:30.246093 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa797c77-8e09-49ae-ba59-d41da90952ca None admin] RBAC: Authorizing `identity:get_user(user_id=360d2cdeb8fb48b89b7ac189977b0daa)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:30.247601 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa797c77-8e09-49ae-ba59-d41da90952ca None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:30.250463 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 155/315] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 24 06:21:30 2022] GET /identity/v3/users/360d2cdeb8fb48b89b7ac189977b0daa => generated 297 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:30.256588 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa05b79-ac15-4f3f-a0ea-ee2c3c42c524 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:30.346804 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:30.368944 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa05b79-ac15-4f3f-a0ea-ee2c3c42c524 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:30.409776 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9daaea-a086-4cd4-b019-e6df17a746cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:30.410757 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9daaea-a086-4cd4-b019-e6df17a746cb None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:30.411228 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9daaea-a086-4cd4-b019-e6df17a746cb None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:30.411691 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9daaea-a086-4cd4-b019-e6df17a746cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:30.485872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9daaea-a086-4cd4-b019-e6df17a746cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:30.487801 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9daaea-a086-4cd4-b019-e6df17a746cb None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:30.574312 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:30.600481 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 161/316] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 24 06:21:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 06:21:30.605891 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa797c77-8e09-49ae-ba59-d41da90952ca None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:30.721964 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa797c77-8e09-49ae-ba59-d41da90952ca None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:30.759830 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00936a6-07a4-4c02-ad96-755d212fd3b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:30.760479 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00936a6-07a4-4c02-ad96-755d212fd3b2 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:30.760766 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00936a6-07a4-4c02-ad96-755d212fd3b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:30.761045 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00936a6-07a4-4c02-ad96-755d212fd3b2 None admin] PATH_INFO: `/v3/projects/c2a0c48ef166451b912daa9e82ddcd09` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:30.762306 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00936a6-07a4-4c02-ad96-755d212fd3b2 None admin] RBAC: Authorizing `identity:get_project(project_id=c2a0c48ef166451b912daa9e82ddcd09)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:30.764902 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00936a6-07a4-4c02-ad96-755d212fd3b2 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:30.766970 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 156/317] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 24 06:21:30 2022] GET /identity/v3/projects/c2a0c48ef166451b912daa9e82ddcd09 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:30.772852 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9daaea-a086-4cd4-b019-e6df17a746cb None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:30.896351 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9daaea-a086-4cd4-b019-e6df17a746cb None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:30.911265 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8358300-18e9-4e4c-98fd-28e741067c58 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:30.911878 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8358300-18e9-4e4c-98fd-28e741067c58 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:30.912371 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8358300-18e9-4e4c-98fd-28e741067c58 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:30.918051 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 157/318] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:21:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 06:21:30.920681 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a7be9a-ebb6-4811-81c4-89a854a25b68 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:30.921221 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a7be9a-ebb6-4811-81c4-89a854a25b68 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:30.921876 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a7be9a-ebb6-4811-81c4-89a854a25b68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:30.936319 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11656d78-bd53-4453-9095-11e1b5e2e909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:30.936933 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11656d78-bd53-4453-9095-11e1b5e2e909 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:30.937235 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11656d78-bd53-4453-9095-11e1b5e2e909 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:30.937512 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11656d78-bd53-4453-9095-11e1b5e2e909 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:30.938247 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11656d78-bd53-4453-9095-11e1b5e2e909 None admin] RBAC: Adding query filter params (role.id=fcfa05f1d2b1474bb4fc20fc988cacd9, scope.project.id=c2a0c48ef166451b912daa9e82ddcd09, user.id=360d2cdeb8fb48b89b7ac189977b0daa) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:30.938556 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11656d78-bd53-4453-9095-11e1b5e2e909 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:30.940008 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11656d78-bd53-4453-9095-11e1b5e2e909 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:30.942691 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-69a7be9a-ebb6-4811-81c4-89a854a25b68 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:21:30.944156 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-69a7be9a-ebb6-4811-81c4-89a854a25b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:21:30.946841 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 162/319] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 24 06:21:30 2022] GET /identity/v3/role_assignments?user.id=360d2cdeb8fb48b89b7ac189977b0daa&scope.project.id=c2a0c48ef166451b912daa9e82ddcd09&role.id=fcfa05f1d2b1474bb4fc20fc988cacd9 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:30.960421 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:31.052839 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 158/320] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:21:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 06:21:31.182282 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e21163-19b2-413e-afcf-01b9503a26af None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:31.183200 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e21163-19b2-413e-afcf-01b9503a26af None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:31.183671 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e21163-19b2-413e-afcf-01b9503a26af None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:31.186191 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 163/321] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:21:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:21:31.192088 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81510109-137e-475b-9e07-398aef29ff6b None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:31.192672 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81510109-137e-475b-9e07-398aef29ff6b None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:31.193139 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81510109-137e-475b-9e07-398aef29ff6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:31.213042 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-81510109-137e-475b-9e07-398aef29ff6b None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:21:31.214481 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-81510109-137e-475b-9e07-398aef29ff6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:21:31.290611 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 159/322] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:21:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:21:31.328500 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e21163-19b2-413e-afcf-01b9503a26af None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:31.379948 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:31.452862 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e21163-19b2-413e-afcf-01b9503a26af None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:31.505661 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ba7f9a-3102-4fde-8fac-6716a9ec8998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:31.506302 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ba7f9a-3102-4fde-8fac-6716a9ec8998 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:31.506611 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ba7f9a-3102-4fde-8fac-6716a9ec8998 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:31.506889 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ba7f9a-3102-4fde-8fac-6716a9ec8998 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:31.508017 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ba7f9a-3102-4fde-8fac-6716a9ec8998 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:31.509486 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ba7f9a-3102-4fde-8fac-6716a9ec8998 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:31.513256 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 164/323] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:21:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:21:31.517022 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81510109-137e-475b-9e07-398aef29ff6b None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:31.570929 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ba7f9a-3102-4fde-8fac-6716a9ec8998 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:31.611684 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:31.635059 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81510109-137e-475b-9e07-398aef29ff6b None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:31.678296 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04229b4d-2faf-4ba9-b63e-09487be80205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:31.678919 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04229b4d-2faf-4ba9-b63e-09487be80205 None admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:31.679197 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04229b4d-2faf-4ba9-b63e-09487be80205 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:31.679503 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04229b4d-2faf-4ba9-b63e-09487be80205 None admin] PATH_INFO: `/v3/groups` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:31.680128 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04229b4d-2faf-4ba9-b63e-09487be80205 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:31.681798 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04229b4d-2faf-4ba9-b63e-09487be80205 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:31.697043 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ba7f9a-3102-4fde-8fac-6716a9ec8998 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:31.708422 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:31.733454 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2541cc82-b5a5-4ce6-b839-a1eedc83823b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:31.737824 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2541cc82-b5a5-4ce6-b839-a1eedc83823b None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:31.737824 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2541cc82-b5a5-4ce6-b839-a1eedc83823b None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:31.737824 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2541cc82-b5a5-4ce6-b839-a1eedc83823b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:31.745069 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2541cc82-b5a5-4ce6-b839-a1eedc83823b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:21:31.751001 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-2541cc82-b5a5-4ce6-b839-a1eedc83823b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 06:21:31.751001 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:21:31.751001 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:21:31.751001 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:21:31.751001 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:21:31.751001 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:21:31.751001 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:21:31.751001 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:21:31.751001 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:21:31.751001 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:21:31.751001 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:21:31.751001 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:21:31.751001 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:21:31.751001 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:21:31.751001 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:21:31.751001 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:21:31.752004 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:21:31.752004 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:21:31.752004 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:21:31.752004 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:21:31.752004 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:21:31.752004 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:21:31.752004 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:21:31.752004 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:21:31.752004 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:21:31.752004 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:21:31.752004 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:21:31.752004 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:21:31.752004 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:21:31.752004 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:21:31.752004 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:21:31.752004 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:21:31.752872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:21:31.752872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:21:31.752872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:21:31.752872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:21:31.752872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:21:31.752872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:21:31.752872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:21:31.752872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:21:31.752872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:21:31.752872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:21:31.752872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 06:21:31.752872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:21:31.752872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 165/324] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 24 06:21:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:21:31.753573 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d66897-c126-4b13-bd3e-bc26d2418894 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:31.753637 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d66897-c126-4b13-bd3e-bc26d2418894 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:31.753907 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d66897-c126-4b13-bd3e-bc26d2418894 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:31.755212 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 166/325] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:21:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:21:31.758683 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d66897-c126-4b13-bd3e-bc26d2418894 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:31.789259 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 160/326] 10.222.0.47 () {60 vars in 1134 bytes} [Wed Aug 24 06:21:31 2022] POST /identity/v3/groups => generated 226 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:21:31.791266 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cca947-628d-432c-a8fd-a32a5b7528d7 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:31.791551 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cca947-628d-432c-a8fd-a32a5b7528d7 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:31.791823 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cca947-628d-432c-a8fd-a32a5b7528d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:31.812189 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-d9cca947-628d-432c-a8fd-a32a5b7528d7 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:21:31.813509 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-d9cca947-628d-432c-a8fd-a32a5b7528d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:21:31.899039 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d66897-c126-4b13-bd3e-bc26d2418894 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:31.899517 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 161/327] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:21:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:21:31.942771 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853ceb1e-2edc-48ea-8683-077f9e9894f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:31.943393 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853ceb1e-2edc-48ea-8683-077f9e9894f1 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:31.943674 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853ceb1e-2edc-48ea-8683-077f9e9894f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:31.943948 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853ceb1e-2edc-48ea-8683-077f9e9894f1 None admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:31.944621 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853ceb1e-2edc-48ea-8683-077f9e9894f1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:31.944922 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853ceb1e-2edc-48ea-8683-077f9e9894f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:31.946591 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853ceb1e-2edc-48ea-8683-077f9e9894f1 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:31.957317 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 167/328] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Aug 24 06:21:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:21:31.962774 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cca947-628d-432c-a8fd-a32a5b7528d7 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:32.046862 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:32.085780 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cca947-628d-432c-a8fd-a32a5b7528d7 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:32.130325 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e2b9ad-ce6d-46b9-b8fa-facddceb2095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:32.130986 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e2b9ad-ce6d-46b9-b8fa-facddceb2095 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:32.131269 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e2b9ad-ce6d-46b9-b8fa-facddceb2095 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:32.131547 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e2b9ad-ce6d-46b9-b8fa-facddceb2095 None admin] PATH_INFO: `/v3/users/b5ba3a8f3deb4f1384a5bd6a62dde7cf` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:32.133130 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e2b9ad-ce6d-46b9-b8fa-facddceb2095 None admin] RBAC: Authorizing `identity:get_user(user_id=b5ba3a8f3deb4f1384a5bd6a62dde7cf)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:32.135187 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e2b9ad-ce6d-46b9-b8fa-facddceb2095 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:32.140105 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 162/329] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 24 06:21:31 2022] GET /identity/v3/users/b5ba3a8f3deb4f1384a5bd6a62dde7cf => generated 293 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:32.142621 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853ceb1e-2edc-48ea-8683-077f9e9894f1 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:32.180762 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e2b9ad-ce6d-46b9-b8fa-facddceb2095 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:32.189443 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:32.294702 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853ceb1e-2edc-48ea-8683-077f9e9894f1 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:32.330537 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e2b9ad-ce6d-46b9-b8fa-facddceb2095 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:32.351539 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b1375c-88b3-46b2-90fa-4bcc2e754de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:32.352767 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b1375c-88b3-46b2-90fa-4bcc2e754de8 None admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:32.353745 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b1375c-88b3-46b2-90fa-4bcc2e754de8 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:32.354344 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b1375c-88b3-46b2-90fa-4bcc2e754de8 None admin] PATH_INFO: `/v3/system/users/b5ba3a8f3deb4f1384a5bd6a62dde7cf/roles/fcfa05f1d2b1474bb4fc20fc988cacd9` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:32.356814 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b1375c-88b3-46b2-90fa-4bcc2e754de8 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=b5ba3a8f3deb4f1384a5bd6a62dde7cf, role_id=fcfa05f1d2b1474bb4fc20fc988cacd9)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:32.359401 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b1375c-88b3-46b2-90fa-4bcc2e754de8 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:32.381094 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e4114e-a6d9-4763-9873-3429939c062a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:32.381784 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e4114e-a6d9-4763-9873-3429939c062a None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:32.382059 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e4114e-a6d9-4763-9873-3429939c062a None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:32.382331 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e4114e-a6d9-4763-9873-3429939c062a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:32.385113 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:32.397004 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e4114e-a6d9-4763-9873-3429939c062a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:21:32.403461 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-d5e4114e-a6d9-4763-9873-3429939c062a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 06:21:32.403461 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:21:32.403461 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:21:32.403461 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:21:32.403461 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:21:32.403461 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:21:32.403461 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:21:32.403461 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:21:32.403461 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:21:32.403461 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:21:32.403461 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:21:32.403461 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:21:32.403461 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:21:32.403461 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:21:32.403461 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:21:32.403461 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:21:32.404428 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:21:32.404428 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:21:32.404428 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:21:32.404428 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:21:32.404428 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:21:32.404428 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:21:32.404428 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:21:32.404428 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:21:32.404428 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:21:32.404428 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:21:32.404428 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:21:32.404428 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:21:32.404428 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:21:32.404428 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:21:32.404428 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:21:32.404428 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:21:32.405272 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:21:32.405272 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:21:32.405272 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:21:32.405272 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:21:32.405272 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:21:32.405272 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:21:32.405272 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:21:32.405272 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:21:32.405272 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:21:32.405272 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:21:32.405272 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 06:21:32.405272 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:21:32.405272 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 163/330] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 24 06:21:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:21:32.412335 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e4114e-a6d9-4763-9873-3429939c062a None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:32.422445 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 168/331] 10.222.0.47 () {58 vars in 1334 bytes} [Wed Aug 24 06:21:32 2022] PUT /identity/v3/system/users/b5ba3a8f3deb4f1384a5bd6a62dde7cf/roles/fcfa05f1d2b1474bb4fc20fc988cacd9 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:21:32.536544 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e4114e-a6d9-4763-9873-3429939c062a None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:32.578194 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519f1611-758e-49a9-ba72-590aaecfbe17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:32.578847 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f1611-758e-49a9-ba72-590aaecfbe17 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:32.579125 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f1611-758e-49a9-ba72-590aaecfbe17 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:32.579440 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f1611-758e-49a9-ba72-590aaecfbe17 None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:32.580131 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519f1611-758e-49a9-ba72-590aaecfbe17 None admin] RBAC: Adding query filter params (name=member) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:32.580432 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519f1611-758e-49a9-ba72-590aaecfbe17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:32.581888 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519f1611-758e-49a9-ba72-590aaecfbe17 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:32.591423 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 164/332] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Aug 24 06:21:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:21:32.597439 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b1375c-88b3-46b2-90fa-4bcc2e754de8 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:32.710790 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:32.723000 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b1375c-88b3-46b2-90fa-4bcc2e754de8 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:32.760744 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc5fef2-c543-4403-acea-83118743a4b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:32.761422 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc5fef2-c543-4403-acea-83118743a4b0 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:32.762109 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc5fef2-c543-4403-acea-83118743a4b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:32.763014 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc5fef2-c543-4403-acea-83118743a4b0 None admin] PATH_INFO: `/v3/users/762c5c56211f40df9a8ba87468d39c05` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:32.764824 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc5fef2-c543-4403-acea-83118743a4b0 None admin] RBAC: Authorizing `identity:get_user(user_id=762c5c56211f40df9a8ba87468d39c05)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:32.766636 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc5fef2-c543-4403-acea-83118743a4b0 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:32.768320 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 169/333] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 24 06:21:32 2022] GET /identity/v3/users/762c5c56211f40df9a8ba87468d39c05 => generated 293 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:32.774412 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519f1611-758e-49a9-ba72-590aaecfbe17 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:32.891973 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:32.948272 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519f1611-758e-49a9-ba72-590aaecfbe17 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:32.996199 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f450a7-5869-4c1e-895f-d39aa3bf9e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:32.996829 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f450a7-5869-4c1e-895f-d39aa3bf9e5e None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:32.997113 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f450a7-5869-4c1e-895f-d39aa3bf9e5e None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:32.997410 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f450a7-5869-4c1e-895f-d39aa3bf9e5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:32.999238 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f450a7-5869-4c1e-895f-d39aa3bf9e5e None admin] RBAC: Adding query filter params (role.id=e1f9b167c51c4a388cbc9594db8a465d, user.id=762c5c56211f40df9a8ba87468d39c05, scope.system=all) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:32.999600 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f450a7-5869-4c1e-895f-d39aa3bf9e5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:33.001067 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f450a7-5869-4c1e-895f-d39aa3bf9e5e None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:33.016751 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 165/334] 10.222.0.47 () {56 vars in 1308 bytes} [Wed Aug 24 06:21:32 2022] GET /identity/v3/role_assignments?user.id=762c5c56211f40df9a8ba87468d39c05&scope.system=all&role.id=e1f9b167c51c4a388cbc9594db8a465d => generated 510 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:34.332280 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01deb655-e0a8-48f5-b9f2-334e92d0d34b None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:34.332814 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01deb655-e0a8-48f5-b9f2-334e92d0d34b None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:34.332814 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01deb655-e0a8-48f5-b9f2-334e92d0d34b None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:34.334510 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 170/335] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:21:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:21:34.339827 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cf9e18-3efb-4d43-abcc-22c9134e0d5c None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:34.340283 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cf9e18-3efb-4d43-abcc-22c9134e0d5c None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:34.340475 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cf9e18-3efb-4d43-abcc-22c9134e0d5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:34.362227 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-d5cf9e18-3efb-4d43-abcc-22c9134e0d5c None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:21:34.363431 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-d5cf9e18-3efb-4d43-abcc-22c9134e0d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:21:34.447474 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 166/336] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:21:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:21:34.687732 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76e6a3d-64af-4832-a7b7-7961e0430754 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:34.688076 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76e6a3d-64af-4832-a7b7-7961e0430754 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:34.688316 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76e6a3d-64af-4832-a7b7-7961e0430754 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:34.690061 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 171/337] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:21:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:21:34.699026 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd3d89d-1539-4927-85ef-030eff0d9167 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:34.699316 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd3d89d-1539-4927-85ef-030eff0d9167 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:34.699584 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd3d89d-1539-4927-85ef-030eff0d9167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:34.723479 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-3dd3d89d-1539-4927-85ef-030eff0d9167 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:21:34.724790 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-3dd3d89d-1539-4927-85ef-030eff0d9167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:21:34.779469 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:34.811120 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 167/338] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:21:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:21:34.970114 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76e6a3d-64af-4832-a7b7-7961e0430754 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:35.119116 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76e6a3d-64af-4832-a7b7-7961e0430754 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:35.152334 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd3e7d6-8a78-4211-8d64-fe9e9d2b3c01 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:35.152650 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd3e7d6-8a78-4211-8d64-fe9e9d2b3c01 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:35.152881 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd3e7d6-8a78-4211-8d64-fe9e9d2b3c01 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:35.154279 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 168/339] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:21:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 06:21:35.166806 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ead268-bb6c-49b1-bb9a-a9680f6050a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:35.167450 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ead268-bb6c-49b1-bb9a-a9680f6050a3 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:35.167725 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ead268-bb6c-49b1-bb9a-a9680f6050a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:35.167994 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ead268-bb6c-49b1-bb9a-a9680f6050a3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:35.171810 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14999fa-4294-44c7-840e-a16fd49e8076 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:35.172100 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14999fa-4294-44c7-840e-a16fd49e8076 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:35.172348 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14999fa-4294-44c7-840e-a16fd49e8076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:35.199077 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ead268-bb6c-49b1-bb9a-a9680f6050a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:21:35.204843 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-d9ead268-bb6c-49b1-bb9a-a9680f6050a3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 06:21:35.204843 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:21:35.204843 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:21:35.204843 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:21:35.204843 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:21:35.204843 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:21:35.204843 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:21:35.204843 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:21:35.204843 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:21:35.204843 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:21:35.204843 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:21:35.204843 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:21:35.204843 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:21:35.204843 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:21:35.204843 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:21:35.204843 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:21:35.205839 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:21:35.205839 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:21:35.205839 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:21:35.205839 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:21:35.205839 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:21:35.205839 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:21:35.205839 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:21:35.205839 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:21:35.205839 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:21:35.205839 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:21:35.205839 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:21:35.205839 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:21:35.205839 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:21:35.205839 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:21:35.205839 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:21:35.205839 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:21:35.206677 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:21:35.206677 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:21:35.206677 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:21:35.206677 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:21:35.206677 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:21:35.206677 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:21:35.206677 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:21:35.206677 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:21:35.206677 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:21:35.206677 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:21:35.206677 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 06:21:35.206677 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:21:35.207323 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-f14999fa-4294-44c7-840e-a16fd49e8076 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:21:35.207896 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-f14999fa-4294-44c7-840e-a16fd49e8076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:21:35.209932 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 172/340] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 24 06:21:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:21:35.212006 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ead268-bb6c-49b1-bb9a-a9680f6050a3 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:35.255637 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:35.355115 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 169/341] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:21:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 06:21:35.355865 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14999fa-4294-44c7-840e-a16fd49e8076 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:35.396433 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ead268-bb6c-49b1-bb9a-a9680f6050a3 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:35.452115 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90f05e2-2291-4e89-982a-d83a8bef40ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:35.453344 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90f05e2-2291-4e89-982a-d83a8bef40ee None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:35.453727 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90f05e2-2291-4e89-982a-d83a8bef40ee None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:35.454146 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90f05e2-2291-4e89-982a-d83a8bef40ee None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:35.470404 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90f05e2-2291-4e89-982a-d83a8bef40ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:21:35.478069 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-e90f05e2-2291-4e89-982a-d83a8bef40ee None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 06:21:35.478069 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:21:35.478069 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:21:35.478069 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:21:35.478069 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:21:35.478069 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:21:35.478069 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:21:35.478069 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:21:35.478069 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:21:35.478069 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:21:35.478069 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:21:35.478069 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:21:35.478069 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:21:35.478069 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:21:35.478069 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:21:35.478069 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:21:35.479216 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:21:35.479216 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:21:35.479216 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:21:35.479216 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:21:35.479216 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:21:35.479216 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:21:35.479216 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:21:35.479216 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:21:35.479216 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:21:35.479216 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:21:35.479216 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:21:35.479216 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:21:35.479216 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:21:35.479216 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:21:35.479216 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:21:35.479216 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:21:35.480051 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:21:35.480051 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:21:35.480051 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:21:35.480051 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:21:35.480051 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:21:35.480051 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:21:35.480051 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:21:35.480051 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:21:35.480051 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:21:35.480051 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:21:35.480051 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 06:21:35.480051 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:21:35.482774 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 173/342] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Aug 24 06:21:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:21:35.484347 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c4e4be-6b96-4cf1-8202-f2111c10b9e2 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:35.484931 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c4e4be-6b96-4cf1-8202-f2111c10b9e2 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:35.489585 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c4e4be-6b96-4cf1-8202-f2111c10b9e2 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:35.491653 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 174/343] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:21:35 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:21:35.498580 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c4e4be-6b96-4cf1-8202-f2111c10b9e2 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:35.514757 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14999fa-4294-44c7-840e-a16fd49e8076 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:35.525596 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:35.566786 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc23a47-7edc-4b0d-bc2e-cc0d49b096dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:35.567225 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc23a47-7edc-4b0d-bc2e-cc0d49b096dd None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:35.567572 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc23a47-7edc-4b0d-bc2e-cc0d49b096dd None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:35.567849 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc23a47-7edc-4b0d-bc2e-cc0d49b096dd None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:35.568557 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc23a47-7edc-4b0d-bc2e-cc0d49b096dd None admin] RBAC: Adding query filter params (name=member) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:35.568860 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc23a47-7edc-4b0d-bc2e-cc0d49b096dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:35.570305 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc23a47-7edc-4b0d-bc2e-cc0d49b096dd None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:35.583960 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 170/344] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Aug 24 06:21:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:35.589266 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7e87b8-f915-4412-a73e-76cec2d7bff0 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:35.589266 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7e87b8-f915-4412-a73e-76cec2d7bff0 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:35.589266 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7e87b8-f915-4412-a73e-76cec2d7bff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:35.608716 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-ac7e87b8-f915-4412-a73e-76cec2d7bff0 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:21:35.610839 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-ac7e87b8-f915-4412-a73e-76cec2d7bff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:21:35.612119 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:35.660277 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c4e4be-6b96-4cf1-8202-f2111c10b9e2 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:35.735110 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c9e9ec-ec86-4161-9718-1deddbe97b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:35.735725 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c9e9ec-ec86-4161-9718-1deddbe97b79 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:35.736002 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c9e9ec-ec86-4161-9718-1deddbe97b79 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:35.736268 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c9e9ec-ec86-4161-9718-1deddbe97b79 None admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:35.736943 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c9e9ec-ec86-4161-9718-1deddbe97b79 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:35.737311 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c9e9ec-ec86-4161-9718-1deddbe97b79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:35.738679 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c9e9ec-ec86-4161-9718-1deddbe97b79 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:35.753319 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 175/345] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 24 06:21:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:21:35.754701 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 171/346] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:21:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:21:35.755265 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c9e9ec-ec86-4161-9718-1deddbe97b79 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:35.755544 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7e87b8-f915-4412-a73e-76cec2d7bff0 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:35.874571 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c9e9ec-ec86-4161-9718-1deddbe97b79 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:35.881042 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:35.885539 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7e87b8-f915-4412-a73e-76cec2d7bff0 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:35.916109 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8614a231-84aa-47b6-86c4-f8820790851d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:35.917105 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8614a231-84aa-47b6-86c4-f8820790851d None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:35.917804 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8614a231-84aa-47b6-86c4-f8820790851d None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:35.918470 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8614a231-84aa-47b6-86c4-f8820790851d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:35.930617 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03272a67-5600-4ca8-962f-a3ebb9a3e7b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:35.931278 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03272a67-5600-4ca8-962f-a3ebb9a3e7b5 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:35.931557 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03272a67-5600-4ca8-962f-a3ebb9a3e7b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:35.931828 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03272a67-5600-4ca8-962f-a3ebb9a3e7b5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:35.938280 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:35.953525 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03272a67-5600-4ca8-962f-a3ebb9a3e7b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:21:35.958558 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-03272a67-5600-4ca8-962f-a3ebb9a3e7b5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 06:21:35.958558 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:21:35.958558 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:21:35.958558 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:21:35.958558 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:21:35.958558 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:21:35.958558 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:21:35.958558 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:21:35.958558 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:21:35.958558 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:21:35.958558 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:21:35.958558 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:21:35.958558 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:21:35.958558 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:21:35.958558 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:21:35.958558 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:21:35.959497 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:21:35.959497 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:21:35.959497 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:21:35.959497 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:21:35.959497 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:21:35.959497 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:21:35.959497 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:21:35.959497 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:21:35.959497 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:21:35.959497 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:21:35.959497 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:21:35.959497 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:21:35.959497 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:21:35.959497 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:21:35.959497 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:21:35.959497 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:21:35.960305 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:21:35.960305 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:21:35.960305 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:21:35.960305 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:21:35.960305 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:21:35.960305 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:21:35.960305 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:21:35.960305 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:21:35.960305 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:21:35.960305 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:21:35.960305 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 06:21:35.960305 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:21:35.960305 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 172/347] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 24 06:21:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:21:35.960974 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03272a67-5600-4ca8-962f-a3ebb9a3e7b5 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:36.004782 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8614a231-84aa-47b6-86c4-f8820790851d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:36.006683 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8614a231-84aa-47b6-86c4-f8820790851d None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:36.102646 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03272a67-5600-4ca8-962f-a3ebb9a3e7b5 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:36.117968 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 176/348] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 24 06:21:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 06:21:36.119156 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8614a231-84aa-47b6-86c4-f8820790851d None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:36.151225 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716c6fa4-6851-4dad-b119-8411f366f416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:36.151791 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716c6fa4-6851-4dad-b119-8411f366f416 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:36.152071 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716c6fa4-6851-4dad-b119-8411f366f416 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:36.152342 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716c6fa4-6851-4dad-b119-8411f366f416 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:36.178741 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:36.193626 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:36.240127 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8614a231-84aa-47b6-86c4-f8820790851d None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:36.255795 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716c6fa4-6851-4dad-b119-8411f366f416 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:36.257551 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716c6fa4-6851-4dad-b119-8411f366f416 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:36.280434 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48208815-9d3b-4f4d-a73b-a22872daf3da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:36.281167 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48208815-9d3b-4f4d-a73b-a22872daf3da None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:36.281548 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48208815-9d3b-4f4d-a73b-a22872daf3da None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:36.281887 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48208815-9d3b-4f4d-a73b-a22872daf3da None admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:36.282618 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48208815-9d3b-4f4d-a73b-a22872daf3da None admin] RBAC: Adding query filter params (name=member) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:36.282984 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48208815-9d3b-4f4d-a73b-a22872daf3da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:36.284531 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48208815-9d3b-4f4d-a73b-a22872daf3da None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:36.295019 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 177/349] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Aug 24 06:21:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:36.296329 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48208815-9d3b-4f4d-a73b-a22872daf3da None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:36.398424 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 173/350] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 24 06:21:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 06:21:36.401429 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bd1b49-eeb1-46c0-8889-e81dbcf599c2 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:36.401429 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bd1b49-eeb1-46c0-8889-e81dbcf599c2 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:36.402050 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bd1b49-eeb1-46c0-8889-e81dbcf599c2 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:36.405715 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 174/351] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:21:36 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:21:36.407108 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bd1b49-eeb1-46c0-8889-e81dbcf599c2 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:36.456794 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48208815-9d3b-4f4d-a73b-a22872daf3da None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:36.476144 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:36.502902 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:36.516498 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b784116c-39ae-456f-aa44-cc6f3247217c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:36.517095 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b784116c-39ae-456f-aa44-cc6f3247217c None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:36.517401 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b784116c-39ae-456f-aa44-cc6f3247217c None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:36.517946 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b784116c-39ae-456f-aa44-cc6f3247217c None admin] PATH_INFO: `/v3/projects/df345274c3b840b486c303c6b301052e` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:36.519334 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b784116c-39ae-456f-aa44-cc6f3247217c None admin] RBAC: Authorizing `identity:get_project(project_id=df345274c3b840b486c303c6b301052e)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:36.521144 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b784116c-39ae-456f-aa44-cc6f3247217c None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:36.523242 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 178/352] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 24 06:21:36 2022] GET /identity/v3/projects/df345274c3b840b486c303c6b301052e => generated 298 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:36.524656 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b784116c-39ae-456f-aa44-cc6f3247217c None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:36.571378 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bd1b49-eeb1-46c0-8889-e81dbcf599c2 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:36.630597 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600afdb9-f587-40bd-a26f-4353d6f299ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:36.631703 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600afdb9-f587-40bd-a26f-4353d6f299ec None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:36.632199 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600afdb9-f587-40bd-a26f-4353d6f299ec None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:36.632722 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600afdb9-f587-40bd-a26f-4353d6f299ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:36.679571 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b784116c-39ae-456f-aa44-cc6f3247217c None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:36.728963 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a91719-0faa-4adb-9393-c8a84bc45fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:36.729910 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a91719-0faa-4adb-9393-c8a84bc45fd5 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:36.730360 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a91719-0faa-4adb-9393-c8a84bc45fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:36.730799 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a91719-0faa-4adb-9393-c8a84bc45fd5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:36.734555 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600afdb9-f587-40bd-a26f-4353d6f299ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:36.736016 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600afdb9-f587-40bd-a26f-4353d6f299ec None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:36.745738 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a91719-0faa-4adb-9393-c8a84bc45fd5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:21:36.752299 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-83a91719-0faa-4adb-9393-c8a84bc45fd5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 06:21:36.752299 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:21:36.752299 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:21:36.752299 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:21:36.752299 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:21:36.752299 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:21:36.752299 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:21:36.752299 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:21:36.752299 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:21:36.752299 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:21:36.752299 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:21:36.752299 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:21:36.752299 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:21:36.752299 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:21:36.752299 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:21:36.752299 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:21:36.753261 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:21:36.753261 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:21:36.753261 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:21:36.753261 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:21:36.753261 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:21:36.753261 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:21:36.753261 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:21:36.753261 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:21:36.753261 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:21:36.753261 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:21:36.753261 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:21:36.753261 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:21:36.753261 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:21:36.753261 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:21:36.753261 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:21:36.753261 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:21:36.754105 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:21:36.754105 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:21:36.754105 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:21:36.754105 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:21:36.754105 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:21:36.754105 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:21:36.754105 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:21:36.754105 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:21:36.754105 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:21:36.754105 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:21:36.754105 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 06:21:36.754105 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:21:36.760550 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 179/353] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 24 06:21:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:21:36.760550 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded10936-a525-4738-8836-f4654f8af88c None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:36.760550 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded10936-a525-4738-8836-f4654f8af88c None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:36.760550 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded10936-a525-4738-8836-f4654f8af88c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:36.782241 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-ded10936-a525-4738-8836-f4654f8af88c None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:21:36.782241 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-ded10936-a525-4738-8836-f4654f8af88c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:21:36.817260 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:36.818803 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:36.874890 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 180/354] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:21:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:21:36.875759 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded10936-a525-4738-8836-f4654f8af88c None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:36.877806 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 175/355] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 24 06:21:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 06:21:36.885260 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600afdb9-f587-40bd-a26f-4353d6f299ec None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:37.011753 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600afdb9-f587-40bd-a26f-4353d6f299ec None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:37.028811 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded10936-a525-4738-8836-f4654f8af88c None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:37.057507 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc4e7eb-7620-4372-890c-4f42211ec6e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:37.058155 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc4e7eb-7620-4372-890c-4f42211ec6e9 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:37.058433 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc4e7eb-7620-4372-890c-4f42211ec6e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:37.058703 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc4e7eb-7620-4372-890c-4f42211ec6e9 None admin] PATH_INFO: `/v3/groups/9fb7ee19cac249789e1821689a52489e` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:37.067293 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc4e7eb-7620-4372-890c-4f42211ec6e9 None admin] RBAC: Authorizing `identity:get_group(group_id=9fb7ee19cac249789e1821689a52489e)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:37.068750 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc4e7eb-7620-4372-890c-4f42211ec6e9 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:37.071574 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 176/356] 10.222.0.47 () {56 vars in 1180 bytes} [Wed Aug 24 06:21:36 2022] GET /identity/v3/groups/9fb7ee19cac249789e1821689a52489e => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:37.071574 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc4e7eb-7620-4372-890c-4f42211ec6e9 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:37.082500 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df43c389-d594-43d5-bf70-a54e9faa67d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:37.083157 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df43c389-d594-43d5-bf70-a54e9faa67d7 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:37.083436 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df43c389-d594-43d5-bf70-a54e9faa67d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:37.083708 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df43c389-d594-43d5-bf70-a54e9faa67d7 None admin] PATH_INFO: `/v3/projects/df345274c3b840b486c303c6b301052e` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:37.085485 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df43c389-d594-43d5-bf70-a54e9faa67d7 None admin] RBAC: Authorizing `identity:get_project(project_id=df345274c3b840b486c303c6b301052e)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:37.087424 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df43c389-d594-43d5-bf70-a54e9faa67d7 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:37.088687 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 181/357] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 24 06:21:36 2022] GET /identity/v3/projects/df345274c3b840b486c303c6b301052e => generated 298 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:37.089595 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df43c389-d594-43d5-bf70-a54e9faa67d7 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:37.122167 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:37.135035 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:37.223507 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df43c389-d594-43d5-bf70-a54e9faa67d7 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:37.228026 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc4e7eb-7620-4372-890c-4f42211ec6e9 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:37.271674 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f5914b-efd0-4445-8d97-7848e7bd98c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:37.271993 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32990627-d7af-41f2-b784-c66ad2e192fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:37.272285 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f5914b-efd0-4445-8d97-7848e7bd98c2 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:37.272567 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f5914b-efd0-4445-8d97-7848e7bd98c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:37.272662 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32990627-d7af-41f2-b784-c66ad2e192fb None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:37.272838 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f5914b-efd0-4445-8d97-7848e7bd98c2 None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:37.272932 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32990627-d7af-41f2-b784-c66ad2e192fb None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:37.273120 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32990627-d7af-41f2-b784-c66ad2e192fb None admin] PATH_INFO: `/v3/projects/c2a0c48ef166451b912daa9e82ddcd09` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:37.273553 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f5914b-efd0-4445-8d97-7848e7bd98c2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:37.273853 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f5914b-efd0-4445-8d97-7848e7bd98c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:37.275403 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32990627-d7af-41f2-b784-c66ad2e192fb None admin] RBAC: Authorizing `identity:get_project(project_id=c2a0c48ef166451b912daa9e82ddcd09)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:37.275511 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f5914b-efd0-4445-8d97-7848e7bd98c2 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:37.276116 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32990627-d7af-41f2-b784-c66ad2e192fb None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:37.278024 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 182/358] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 24 06:21:37 2022] GET /identity/v3/projects/c2a0c48ef166451b912daa9e82ddcd09 => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:37.280482 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68092f50-ef08-4230-9941-2a44bd42f9c0 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:37.280979 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68092f50-ef08-4230-9941-2a44bd42f9c0 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:37.281495 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68092f50-ef08-4230-9941-2a44bd42f9c0 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:37.283638 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 183/359] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:21:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:21:37.284397 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68092f50-ef08-4230-9941-2a44bd42f9c0 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:37.293528 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 177/360] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Aug 24 06:21:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:37.294960 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f5914b-efd0-4445-8d97-7848e7bd98c2 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:37.407612 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:37.420008 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68092f50-ef08-4230-9941-2a44bd42f9c0 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:37.434398 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:37.434530 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f5914b-efd0-4445-8d97-7848e7bd98c2 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:37.465647 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1061dbb8-9b17-4e40-9fe8-3bec79dfcb5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:37.471413 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1061dbb8-9b17-4e40-9fe8-3bec79dfcb5e None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:37.471413 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1061dbb8-9b17-4e40-9fe8-3bec79dfcb5e None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:37.471413 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1061dbb8-9b17-4e40-9fe8-3bec79dfcb5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:37.471413 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1061dbb8-9b17-4e40-9fe8-3bec79dfcb5e None admin] RBAC: Adding query filter params (group.id=9fb7ee19cac249789e1821689a52489e, role.id=e1f9b167c51c4a388cbc9594db8a465d, scope.project.id=df345274c3b840b486c303c6b301052e) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:37.471413 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1061dbb8-9b17-4e40-9fe8-3bec79dfcb5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:37.471413 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1061dbb8-9b17-4e40-9fe8-3bec79dfcb5e None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:37.471834 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75972124-c568-4452-b126-589e290d9db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:37.471834 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75972124-c568-4452-b126-589e290d9db0 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:37.471834 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75972124-c568-4452-b126-589e290d9db0 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:37.471834 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75972124-c568-4452-b126-589e290d9db0 None admin] PATH_INFO: `/v3/groups/9fb7ee19cac249789e1821689a52489e` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:37.473142 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75972124-c568-4452-b126-589e290d9db0 None admin] RBAC: Authorizing `identity:get_group(group_id=9fb7ee19cac249789e1821689a52489e)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:37.474609 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75972124-c568-4452-b126-589e290d9db0 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:37.476320 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 178/361] 10.222.0.47 () {56 vars in 1180 bytes} [Wed Aug 24 06:21:37 2022] GET /identity/v3/groups/9fb7ee19cac249789e1821689a52489e => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:37.476740 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 184/362] 10.222.0.47 () {56 vars in 1376 bytes} [Wed Aug 24 06:21:37 2022] GET /identity/v3/role_assignments?group.id=9fb7ee19cac249789e1821689a52489e&scope.project.id=df345274c3b840b486c303c6b301052e&role.id=e1f9b167c51c4a388cbc9594db8a465d => generated 260 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:37.476993 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75972124-c568-4452-b126-589e290d9db0 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:37.478028 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1061dbb8-9b17-4e40-9fe8-3bec79dfcb5e None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:37.629541 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1061dbb8-9b17-4e40-9fe8-3bec79dfcb5e None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:37.647462 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75972124-c568-4452-b126-589e290d9db0 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:37.691461 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b247b7-8075-4384-9583-2303df6e04e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:37.692106 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b247b7-8075-4384-9583-2303df6e04e6 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:37.692392 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b247b7-8075-4384-9583-2303df6e04e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:37.692664 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b247b7-8075-4384-9583-2303df6e04e6 None admin] PATH_INFO: `/v3/groups/9fb7ee19cac249789e1821689a52489e` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:37.697422 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b247b7-8075-4384-9583-2303df6e04e6 None admin] RBAC: Authorizing `identity:get_group(group_id=9fb7ee19cac249789e1821689a52489e)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:37.699644 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b247b7-8075-4384-9583-2303df6e04e6 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:37.701894 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 185/363] 10.222.0.47 () {56 vars in 1180 bytes} [Wed Aug 24 06:21:37 2022] GET /identity/v3/groups/9fb7ee19cac249789e1821689a52489e => generated 226 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:37.703103 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fee088-e833-4622-b7bd-b9862b4704e8 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:37.703383 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fee088-e833-4622-b7bd-b9862b4704e8 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:37.703655 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fee088-e833-4622-b7bd-b9862b4704e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:37.705091 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4243677-2e23-4cb1-af1c-3a6e60c381dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:37.705783 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4243677-2e23-4cb1-af1c-3a6e60c381dc None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:37.706060 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4243677-2e23-4cb1-af1c-3a6e60c381dc None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:37.706430 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4243677-2e23-4cb1-af1c-3a6e60c381dc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:37.727264 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4243677-2e23-4cb1-af1c-3a6e60c381dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:21:37.731428 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-29fee088-e833-4622-b7bd-b9862b4704e8 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:21:37.732686 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-a4243677-2e23-4cb1-af1c-3a6e60c381dc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 06:21:37.732686 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:21:37.732686 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:21:37.732686 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:21:37.732686 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:21:37.732686 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:21:37.732686 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:21:37.732686 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:21:37.732686 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:21:37.732686 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:21:37.732686 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:21:37.732686 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:21:37.732686 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:21:37.732686 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:21:37.732686 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:21:37.732686 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:21:37.734860 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:21:37.734860 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:21:37.734860 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:21:37.734860 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:21:37.734860 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:21:37.734860 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:21:37.734860 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:21:37.734860 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:21:37.734860 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:21:37.734860 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:21:37.734860 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:21:37.734860 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:21:37.734860 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:21:37.734860 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:21:37.734860 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:21:37.734860 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:21:37.737350 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:21:37.737350 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:21:37.737350 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:21:37.737350 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:21:37.737350 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:21:37.737350 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:21:37.737350 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:21:37.737350 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:21:37.737350 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:21:37.737350 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:21:37.737350 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 06:21:37.737350 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:21:37.738013 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-29fee088-e833-4622-b7bd-b9862b4704e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:21:37.738093 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 179/364] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Aug 24 06:21:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:21:37.738093 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4243677-2e23-4cb1-af1c-3a6e60c381dc None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:37.758534 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:37.777697 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:37.839057 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 186/365] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:21:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:21:37.840088 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fee088-e833-4622-b7bd-b9862b4704e8 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:37.884062 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4243677-2e23-4cb1-af1c-3a6e60c381dc None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:37.935170 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0690a1fe-fa99-49d4-b5c2-2bd233c5b527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:37.936078 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0690a1fe-fa99-49d4-b5c2-2bd233c5b527 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:37.936361 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0690a1fe-fa99-49d4-b5c2-2bd233c5b527 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:37.936634 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0690a1fe-fa99-49d4-b5c2-2bd233c5b527 None admin] PATH_INFO: `/v3/users/b5ba3a8f3deb4f1384a5bd6a62dde7cf` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:37.938396 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0690a1fe-fa99-49d4-b5c2-2bd233c5b527 None admin] RBAC: Authorizing `identity:get_user(user_id=b5ba3a8f3deb4f1384a5bd6a62dde7cf)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:37.940509 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0690a1fe-fa99-49d4-b5c2-2bd233c5b527 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:37.943242 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 180/366] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 24 06:21:37 2022] GET /identity/v3/users/b5ba3a8f3deb4f1384a5bd6a62dde7cf => generated 293 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:37.943968 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0690a1fe-fa99-49d4-b5c2-2bd233c5b527 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:37.976614 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fee088-e833-4622-b7bd-b9862b4704e8 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:38.029979 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673eb2f0-15fd-44a1-9b25-0a77c0c8a4b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:38.030595 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673eb2f0-15fd-44a1-9b25-0a77c0c8a4b3 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:38.030981 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673eb2f0-15fd-44a1-9b25-0a77c0c8a4b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:38.031263 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673eb2f0-15fd-44a1-9b25-0a77c0c8a4b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:38.032118 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673eb2f0-15fd-44a1-9b25-0a77c0c8a4b3 None admin] RBAC: Adding query filter params (group.id=9fb7ee19cac249789e1821689a52489e, role.id=b71c87e12dc1452ea3c99a7bf28585ca, scope.project.id=df345274c3b840b486c303c6b301052e) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:38.032432 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673eb2f0-15fd-44a1-9b25-0a77c0c8a4b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:38.034011 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673eb2f0-15fd-44a1-9b25-0a77c0c8a4b3 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:38.040142 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 187/367] 10.222.0.47 () {56 vars in 1376 bytes} [Wed Aug 24 06:21:37 2022] GET /identity/v3/role_assignments?group.id=9fb7ee19cac249789e1821689a52489e&scope.project.id=df345274c3b840b486c303c6b301052e&role.id=b71c87e12dc1452ea3c99a7bf28585ca => generated 260 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:38.041052 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673eb2f0-15fd-44a1-9b25-0a77c0c8a4b3 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:38.043414 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:38.066922 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0690a1fe-fa99-49d4-b5c2-2bd233c5b527 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:38.078069 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:38.106105 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af71ed04-dc64-4536-9f91-8d3024dbdd11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:38.107510 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af71ed04-dc64-4536-9f91-8d3024dbdd11 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:38.107802 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af71ed04-dc64-4536-9f91-8d3024dbdd11 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:38.108137 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af71ed04-dc64-4536-9f91-8d3024dbdd11 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:38.108906 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af71ed04-dc64-4536-9f91-8d3024dbdd11 None admin] RBAC: Adding query filter params (group.id=9fb7ee19cac249789e1821689a52489e, role.id=e1f9b167c51c4a388cbc9594db8a465d, scope.project.id=c2a0c48ef166451b912daa9e82ddcd09) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:38.109231 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af71ed04-dc64-4536-9f91-8d3024dbdd11 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:38.110688 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af71ed04-dc64-4536-9f91-8d3024dbdd11 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:38.117244 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 181/368] 10.222.0.47 () {56 vars in 1376 bytes} [Wed Aug 24 06:21:37 2022] GET /identity/v3/role_assignments?group.id=9fb7ee19cac249789e1821689a52489e&scope.project.id=c2a0c48ef166451b912daa9e82ddcd09&role.id=e1f9b167c51c4a388cbc9594db8a465d => generated 260 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:38.118982 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af71ed04-dc64-4536-9f91-8d3024dbdd11 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:38.172880 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673eb2f0-15fd-44a1-9b25-0a77c0c8a4b3 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:38.215249 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa149e7-5df7-416a-b5bb-9e67aa4ea29b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:38.215995 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa149e7-5df7-416a-b5bb-9e67aa4ea29b None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:38.216820 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa149e7-5df7-416a-b5bb-9e67aa4ea29b None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:38.217514 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa149e7-5df7-416a-b5bb-9e67aa4ea29b None admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:38.218406 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa149e7-5df7-416a-b5bb-9e67aa4ea29b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:38.218717 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa149e7-5df7-416a-b5bb-9e67aa4ea29b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:38.220264 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa149e7-5df7-416a-b5bb-9e67aa4ea29b None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:38.230483 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 188/369] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 24 06:21:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:38.250178 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af71ed04-dc64-4536-9f91-8d3024dbdd11 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:38.255454 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa149e7-5df7-416a-b5bb-9e67aa4ea29b None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:38.292871 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739d1017-93a1-437b-b4a0-48a01b241276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:38.293801 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739d1017-93a1-437b-b4a0-48a01b241276 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:38.294964 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739d1017-93a1-437b-b4a0-48a01b241276 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:38.295384 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739d1017-93a1-437b-b4a0-48a01b241276 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:38.296783 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739d1017-93a1-437b-b4a0-48a01b241276 None admin] RBAC: Adding query filter params (role.id=fcfa05f1d2b1474bb4fc20fc988cacd9, user.id=b5ba3a8f3deb4f1384a5bd6a62dde7cf, scope.system=all) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:38.297098 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739d1017-93a1-437b-b4a0-48a01b241276 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:38.298776 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739d1017-93a1-437b-b4a0-48a01b241276 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:38.313557 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 182/370] 10.222.0.47 () {56 vars in 1308 bytes} [Wed Aug 24 06:21:38 2022] GET /identity/v3/role_assignments?user.id=b5ba3a8f3deb4f1384a5bd6a62dde7cf&scope.system=all&role.id=fcfa05f1d2b1474bb4fc20fc988cacd9 => generated 510 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:38.344070 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739d1017-93a1-437b-b4a0-48a01b241276 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:38.348816 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:38.450014 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:38.473838 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa149e7-5df7-416a-b5bb-9e67aa4ea29b None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:38.539764 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddecc7ae-f4f5-4009-aa1c-ce3295ff9b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:38.540499 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddecc7ae-f4f5-4009-aa1c-ce3295ff9b83 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:38.540778 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddecc7ae-f4f5-4009-aa1c-ce3295ff9b83 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:38.541052 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddecc7ae-f4f5-4009-aa1c-ce3295ff9b83 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:38.584233 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739d1017-93a1-437b-b4a0-48a01b241276 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:38.664627 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddecc7ae-f4f5-4009-aa1c-ce3295ff9b83 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:38.666502 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddecc7ae-f4f5-4009-aa1c-ce3295ff9b83 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:38.682482 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11857ecd-6bcc-4139-9f09-a2a1c04078c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:38.683119 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11857ecd-6bcc-4139-9f09-a2a1c04078c0 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:38.683402 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11857ecd-6bcc-4139-9f09-a2a1c04078c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:38.683683 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11857ecd-6bcc-4139-9f09-a2a1c04078c0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:38.718431 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11857ecd-6bcc-4139-9f09-a2a1c04078c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:21:38.723741 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-11857ecd-6bcc-4139-9f09-a2a1c04078c0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:21:38.723741 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:21:38.723741 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:21:38.723741 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:21:38.723741 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:21:38.723741 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:21:38.723741 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:21:38.723741 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:21:38.723741 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:21:38.723741 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:21:38.723741 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:21:38.723741 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:21:38.723741 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:21:38.723741 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:21:38.723741 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:21:38.723741 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:21:38.724670 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:21:38.724670 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:21:38.724670 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:21:38.724670 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:21:38.724670 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:21:38.724670 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:21:38.724670 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:21:38.724670 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:21:38.724670 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:21:38.724670 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:21:38.724670 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:21:38.724670 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:21:38.724670 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:21:38.724670 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:21:38.724670 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:21:38.724670 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:21:38.725507 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:21:38.725507 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:21:38.725507 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:21:38.725507 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:21:38.725507 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:21:38.725507 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:21:38.725507 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:21:38.725507 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:21:38.725507 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:21:38.725507 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:21:38.725507 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:21:38.725507 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:21:38.725507 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 183/371] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Aug 24 06:21:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 381 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:21:38.730587 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11857ecd-6bcc-4139-9f09-a2a1c04078c0 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:38.772920 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:38.786057 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 189/372] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 24 06:21:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 531 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 06:21:38.793047 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddecc7ae-f4f5-4009-aa1c-ce3295ff9b83 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:38.848370 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:38.926111 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11857ecd-6bcc-4139-9f09-a2a1c04078c0 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:38.977105 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39c14cb-fcca-45b6-8160-0785ff60dcc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:38.977689 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39c14cb-fcca-45b6-8160-0785ff60dcc1 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:38.977969 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39c14cb-fcca-45b6-8160-0785ff60dcc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:38.978246 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39c14cb-fcca-45b6-8160-0785ff60dcc1 None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:38.978933 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39c14cb-fcca-45b6-8160-0785ff60dcc1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:38.979242 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39c14cb-fcca-45b6-8160-0785ff60dcc1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:38.980904 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39c14cb-fcca-45b6-8160-0785ff60dcc1 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:38.998697 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 184/373] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 24 06:21:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:39.006311 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39c14cb-fcca-45b6-8160-0785ff60dcc1 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:39.006642 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddecc7ae-f4f5-4009-aa1c-ce3295ff9b83 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:39.102000 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d854200-5d16-4d79-8319-b6592b812f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:39.102655 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d854200-5d16-4d79-8319-b6592b812f99 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:39.102949 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d854200-5d16-4d79-8319-b6592b812f99 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:39.103230 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d854200-5d16-4d79-8319-b6592b812f99 None admin] PATH_INFO: `/v3/projects/c2a0c48ef166451b912daa9e82ddcd09` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:39.104502 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d854200-5d16-4d79-8319-b6592b812f99 None admin] RBAC: Authorizing `identity:get_project(project_id=c2a0c48ef166451b912daa9e82ddcd09)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:39.105972 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d854200-5d16-4d79-8319-b6592b812f99 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:39.108470 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 190/374] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 24 06:21:38 2022] GET /identity/v3/projects/c2a0c48ef166451b912daa9e82ddcd09 => generated 302 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:21:39.118241 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d854200-5d16-4d79-8319-b6592b812f99 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:39.177940 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39c14cb-fcca-45b6-8160-0785ff60dcc1 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:39.225316 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:39.228785 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:39.249638 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091123e0-ecd8-4f2e-a730-5b749a589abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:39.250263 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091123e0-ecd8-4f2e-a730-5b749a589abf None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:39.250538 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091123e0-ecd8-4f2e-a730-5b749a589abf None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:39.250808 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091123e0-ecd8-4f2e-a730-5b749a589abf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:39.299429 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d854200-5d16-4d79-8319-b6592b812f99 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:39.352479 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091123e0-ecd8-4f2e-a730-5b749a589abf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:39.354485 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091123e0-ecd8-4f2e-a730-5b749a589abf None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:39.368911 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e9d5f9-31ee-477d-a071-ae58365aeead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:39.370070 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e9d5f9-31ee-477d-a071-ae58365aeead None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:39.370565 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e9d5f9-31ee-477d-a071-ae58365aeead None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:39.371045 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e9d5f9-31ee-477d-a071-ae58365aeead None admin] PATH_INFO: `/v3/groups/9fb7ee19cac249789e1821689a52489e` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:39.374528 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e9d5f9-31ee-477d-a071-ae58365aeead None admin] RBAC: Authorizing `identity:get_group(group_id=9fb7ee19cac249789e1821689a52489e)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:39.376345 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e9d5f9-31ee-477d-a071-ae58365aeead None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:39.378594 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 191/375] 10.222.0.47 () {56 vars in 1180 bytes} [Wed Aug 24 06:21:39 2022] GET /identity/v3/groups/9fb7ee19cac249789e1821689a52489e => generated 226 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:39.385357 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e9d5f9-31ee-477d-a071-ae58365aeead None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:39.483445 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 185/376] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 24 06:21:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 06:21:39.490324 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091123e0-ecd8-4f2e-a730-5b749a589abf None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:39.524608 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:39.596923 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e9d5f9-31ee-477d-a071-ae58365aeead None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:39.621889 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091123e0-ecd8-4f2e-a730-5b749a589abf None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:39.658426 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:39.675190 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e044f869-d1fd-494a-b169-cc18c5cafda7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:39.676084 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e044f869-d1fd-494a-b169-cc18c5cafda7 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:39.676604 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e044f869-d1fd-494a-b169-cc18c5cafda7 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:39.677024 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e044f869-d1fd-494a-b169-cc18c5cafda7 None admin] PATH_INFO: `/v3/projects/24c1e9e70e07496595a602ca047e6a1f` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:39.678572 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e044f869-d1fd-494a-b169-cc18c5cafda7 None admin] RBAC: Authorizing `identity:get_project(project_id=24c1e9e70e07496595a602ca047e6a1f)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:39.678990 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3c0ae8-32d4-4143-9c19-4a9abc39aa18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:39.679713 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3c0ae8-32d4-4143-9c19-4a9abc39aa18 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:39.680124 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3c0ae8-32d4-4143-9c19-4a9abc39aa18 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:39.680494 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3c0ae8-32d4-4143-9c19-4a9abc39aa18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:39.680901 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e044f869-d1fd-494a-b169-cc18c5cafda7 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:39.681375 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3c0ae8-32d4-4143-9c19-4a9abc39aa18 None admin] RBAC: Adding query filter params (group.id=9fb7ee19cac249789e1821689a52489e, role.id=b71c87e12dc1452ea3c99a7bf28585ca, scope.project.id=c2a0c48ef166451b912daa9e82ddcd09) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:39.681685 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3c0ae8-32d4-4143-9c19-4a9abc39aa18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:39.683130 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3c0ae8-32d4-4143-9c19-4a9abc39aa18 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:39.684123 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 186/377] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 24 06:21:39 2022] GET /identity/v3/projects/24c1e9e70e07496595a602ca047e6a1f => generated 344 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:39.690296 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e044f869-d1fd-494a-b169-cc18c5cafda7 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:39.699469 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 192/378] 10.222.0.47 () {56 vars in 1376 bytes} [Wed Aug 24 06:21:39 2022] GET /identity/v3/role_assignments?group.id=9fb7ee19cac249789e1821689a52489e&scope.project.id=c2a0c48ef166451b912daa9e82ddcd09&role.id=b71c87e12dc1452ea3c99a7bf28585ca => generated 260 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:21:39.834062 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:39.844247 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e044f869-d1fd-494a-b169-cc18c5cafda7 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:39.893824 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a3e531-9fe5-49c2-9d49-bb3e7857d909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:39.894474 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a3e531-9fe5-49c2-9d49-bb3e7857d909 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:39.894819 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a3e531-9fe5-49c2-9d49-bb3e7857d909 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:39.895099 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a3e531-9fe5-49c2-9d49-bb3e7857d909 None admin] PATH_INFO: `/v3/groups/2faefd426a0748d6a8263d0eb0102204` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:39.900409 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a3e531-9fe5-49c2-9d49-bb3e7857d909 None admin] RBAC: Authorizing `identity:get_group(group_id=2faefd426a0748d6a8263d0eb0102204)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:39.902159 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a3e531-9fe5-49c2-9d49-bb3e7857d909 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:39.904219 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 187/379] 10.222.0.47 () {56 vars in 1180 bytes} [Wed Aug 24 06:21:39 2022] GET /identity/v3/groups/2faefd426a0748d6a8263d0eb0102204 => generated 229 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:21:39.910280 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3c0ae8-32d4-4143-9c19-4a9abc39aa18 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:40.049779 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3c0ae8-32d4-4143-9c19-4a9abc39aa18 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:40.098796 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402dc969-325f-4a7b-958e-f7368b1478a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:40.099338 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402dc969-325f-4a7b-958e-f7368b1478a0 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:40.099615 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402dc969-325f-4a7b-958e-f7368b1478a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:40.099884 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402dc969-325f-4a7b-958e-f7368b1478a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:40.100617 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402dc969-325f-4a7b-958e-f7368b1478a0 None admin] RBAC: Adding query filter params (group.id=2faefd426a0748d6a8263d0eb0102204, role.id=486144ad1bcd417e86acdbc1d14327bf, scope.project.id=24c1e9e70e07496595a602ca047e6a1f) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:40.100982 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402dc969-325f-4a7b-958e-f7368b1478a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:40.102464 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402dc969-325f-4a7b-958e-f7368b1478a0 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:40.113802 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 193/380] 10.222.0.47 () {56 vars in 1376 bytes} [Wed Aug 24 06:21:39 2022] GET /identity/v3/role_assignments?group.id=2faefd426a0748d6a8263d0eb0102204&scope.project.id=24c1e9e70e07496595a602ca047e6a1f&role.id=486144ad1bcd417e86acdbc1d14327bf => generated 260 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:40.211477 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b90c493-55ba-499b-8270-eab5503ee96d None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:40.211846 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b90c493-55ba-499b-8270-eab5503ee96d None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:40.212100 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b90c493-55ba-499b-8270-eab5503ee96d None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:40.213780 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 188/381] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:21:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:21:40.220658 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7886b49e-e5f8-4bca-bf39-ea984bebdc9a None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:40.220984 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7886b49e-e5f8-4bca-bf39-ea984bebdc9a None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:40.221290 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7886b49e-e5f8-4bca-bf39-ea984bebdc9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:40.250721 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-7886b49e-e5f8-4bca-bf39-ea984bebdc9a None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:21:40.251992 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-7886b49e-e5f8-4bca-bf39-ea984bebdc9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:21:40.327382 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:40.352539 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 194/382] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:21:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 06:21:40.678347 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b90c493-55ba-499b-8270-eab5503ee96d None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:40.837895 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b90c493-55ba-499b-8270-eab5503ee96d None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:40.887535 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d58523-cd06-4e6a-853e-511a55366769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:40.888350 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d58523-cd06-4e6a-853e-511a55366769 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:40.888822 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d58523-cd06-4e6a-853e-511a55366769 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:40.889103 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d58523-cd06-4e6a-853e-511a55366769 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:40.908505 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d58523-cd06-4e6a-853e-511a55366769 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:21:40.913436 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-75d58523-cd06-4e6a-853e-511a55366769 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 06:21:40.913436 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:21:40.913436 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:21:40.913436 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:21:40.913436 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:21:40.913436 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:21:40.913436 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:21:40.913436 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:21:40.913436 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:21:40.913436 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:21:40.913436 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:21:40.913436 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:21:40.913436 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:21:40.913436 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:21:40.913436 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:21:40.913436 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:21:40.914330 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:21:40.914330 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:21:40.914330 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:21:40.914330 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:21:40.914330 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:21:40.914330 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:21:40.914330 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:21:40.914330 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:21:40.914330 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:21:40.914330 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:21:40.914330 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:21:40.914330 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:21:40.914330 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:21:40.914330 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:21:40.914330 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:21:40.914330 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:21:40.915719 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:21:40.915719 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:21:40.915719 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:21:40.915719 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:21:40.915719 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:21:40.915719 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:21:40.915719 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:21:40.915719 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:21:40.915719 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:21:40.915719 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:21:40.915719 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 06:21:40.915719 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:21:40.915719 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 189/383] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 24 06:21:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:21:40.921425 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7886b49e-e5f8-4bca-bf39-ea984bebdc9a None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:41.030235 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9dc880-3026-4a42-b152-314408652901 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:41.030545 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9dc880-3026-4a42-b152-314408652901 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:41.030809 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9dc880-3026-4a42-b152-314408652901 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:41.032340 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 190/384] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:21:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:21:41.039021 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93187c6-a64a-417c-a9ac-8ed03c330bc7 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:41.039510 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93187c6-a64a-417c-a9ac-8ed03c330bc7 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:41.039995 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93187c6-a64a-417c-a9ac-8ed03c330bc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:41.069348 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-d93187c6-a64a-417c-a9ac-8ed03c330bc7 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:21:41.071049 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-d93187c6-a64a-417c-a9ac-8ed03c330bc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:21:41.086245 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:41.091162 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7886b49e-e5f8-4bca-bf39-ea984bebdc9a None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:41.138713 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaca257-cadc-4e71-a5d4-16f750f187eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:41.139629 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaca257-cadc-4e71-a5d4-16f750f187eb None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:41.140015 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaca257-cadc-4e71-a5d4-16f750f187eb None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:41.140298 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaca257-cadc-4e71-a5d4-16f750f187eb None admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:41.140984 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaca257-cadc-4e71-a5d4-16f750f187eb None admin] RBAC: Adding query filter params (name=member) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:41.141310 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaca257-cadc-4e71-a5d4-16f750f187eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:41.142865 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaca257-cadc-4e71-a5d4-16f750f187eb None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:41.154961 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 195/385] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Aug 24 06:21:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:41.156527 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b622cfb4-d1fd-4a4c-841c-4d883af4dbe5 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:41.156952 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b622cfb4-d1fd-4a4c-841c-4d883af4dbe5 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:41.157341 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b622cfb4-d1fd-4a4c-841c-4d883af4dbe5 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:41.158695 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 196/386] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:21:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:21:41.163864 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b622cfb4-d1fd-4a4c-841c-4d883af4dbe5 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:41.166829 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 191/387] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:21:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:21:41.169855 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ad51fc-6a22-47da-9ef3-6b8a1b46af45 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:41.170148 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ad51fc-6a22-47da-9ef3-6b8a1b46af45 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:41.170528 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ad51fc-6a22-47da-9ef3-6b8a1b46af45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:41.194083 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-14ad51fc-6a22-47da-9ef3-6b8a1b46af45 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:21:41.195562 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-14ad51fc-6a22-47da-9ef3-6b8a1b46af45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:21:41.221609 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:41.292355 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 192/388] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:21:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:21:41.317882 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b622cfb4-d1fd-4a4c-841c-4d883af4dbe5 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:41.372952 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3d7ea4-47ca-42a0-9ac9-0e22b8c87f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:41.373885 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3d7ea4-47ca-42a0-9ac9-0e22b8c87f51 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:41.374171 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3d7ea4-47ca-42a0-9ac9-0e22b8c87f51 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:41.374455 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3d7ea4-47ca-42a0-9ac9-0e22b8c87f51 None admin] PATH_INFO: `/v3/groups/9fb7ee19cac249789e1821689a52489e` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:41.378683 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3d7ea4-47ca-42a0-9ac9-0e22b8c87f51 None admin] RBAC: Authorizing `identity:get_group(group_id=9fb7ee19cac249789e1821689a52489e)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:41.380148 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3d7ea4-47ca-42a0-9ac9-0e22b8c87f51 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:41.381862 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 197/389] 10.222.0.47 () {56 vars in 1180 bytes} [Wed Aug 24 06:21:41 2022] GET /identity/v3/groups/9fb7ee19cac249789e1821689a52489e => generated 226 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:41.388449 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ad51fc-6a22-47da-9ef3-6b8a1b46af45 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:41.506402 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:41.535928 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ad51fc-6a22-47da-9ef3-6b8a1b46af45 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:41.579730 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aaaa86-86fa-4245-80b6-72048d2563ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:41.582708 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aaaa86-86fa-4245-80b6-72048d2563ca None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:41.582708 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aaaa86-86fa-4245-80b6-72048d2563ca None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:41.582708 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aaaa86-86fa-4245-80b6-72048d2563ca None admin] PATH_INFO: `/v3/projects/df345274c3b840b486c303c6b301052e` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:41.582708 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16aaaa86-86fa-4245-80b6-72048d2563ca None admin] RBAC: Authorizing `identity:get_project(project_id=df345274c3b840b486c303c6b301052e)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:41.584415 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16aaaa86-86fa-4245-80b6-72048d2563ca None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:41.585684 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 193/390] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 24 06:21:41 2022] GET /identity/v3/projects/df345274c3b840b486c303c6b301052e => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:41.594494 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3d7ea4-47ca-42a0-9ac9-0e22b8c87f51 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:41.656123 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aaaa86-86fa-4245-80b6-72048d2563ca None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:41.749223 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3d7ea4-47ca-42a0-9ac9-0e22b8c87f51 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:41.764415 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:41.786654 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87a5255-3b5a-4850-8ade-c76e7b124c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:41.787421 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87a5255-3b5a-4850-8ade-c76e7b124c21 None admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:41.787778 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87a5255-3b5a-4850-8ade-c76e7b124c21 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:41.788063 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87a5255-3b5a-4850-8ade-c76e7b124c21 None admin] PATH_INFO: `/v3/projects/df345274c3b840b486c303c6b301052e/groups/9fb7ee19cac249789e1821689a52489e/roles/e1f9b167c51c4a388cbc9594db8a465d` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:41.788474 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aaaa86-86fa-4245-80b6-72048d2563ca None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:41.790004 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87a5255-3b5a-4850-8ade-c76e7b124c21 None admin] RBAC: Authorizing `identity:create_grant(project_id=df345274c3b840b486c303c6b301052e, group_id=9fb7ee19cac249789e1821689a52489e, role_id=e1f9b167c51c4a388cbc9594db8a465d)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:41.791445 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87a5255-3b5a-4850-8ade-c76e7b124c21 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:41.830456 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c0e9ff-0374-400e-8ee5-fbc8491a4b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:41.831102 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c0e9ff-0374-400e-8ee5-fbc8491a4b0c None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:41.831383 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c0e9ff-0374-400e-8ee5-fbc8491a4b0c None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:41.831660 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c0e9ff-0374-400e-8ee5-fbc8491a4b0c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:41.849440 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c0e9ff-0374-400e-8ee5-fbc8491a4b0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:21:41.854378 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-f4c0e9ff-0374-400e-8ee5-fbc8491a4b0c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 06:21:41.854378 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:21:41.854378 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:21:41.854378 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:21:41.854378 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:21:41.854378 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:21:41.854378 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:21:41.854378 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:21:41.854378 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:21:41.854378 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:21:41.854378 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:21:41.854378 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:21:41.854378 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:21:41.854378 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:21:41.854378 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:21:41.854378 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:21:41.855603 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:21:41.855603 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:21:41.855603 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:21:41.855603 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:21:41.855603 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:21:41.855603 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:21:41.855603 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:21:41.855603 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:21:41.855603 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:21:41.855603 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:21:41.855603 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:21:41.855603 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:21:41.855603 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:21:41.855603 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:21:41.855603 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:21:41.855603 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:21:41.857023 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:21:41.857023 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:21:41.857023 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:21:41.857023 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:21:41.857023 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:21:41.857023 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:21:41.857023 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:21:41.857023 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:21:41.857023 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:21:41.857023 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:21:41.857023 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 06:21:41.857023 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:21:41.857023 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 194/391] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Aug 24 06:21:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:21:41.857996 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c0e9ff-0374-400e-8ee5-fbc8491a4b0c None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:41.865390 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:41.909371 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 198/392] 10.222.0.47 () {58 vars in 1442 bytes} [Wed Aug 24 06:21:41 2022] PUT /identity/v3/projects/df345274c3b840b486c303c6b301052e/groups/9fb7ee19cac249789e1821689a52489e/roles/e1f9b167c51c4a388cbc9594db8a465d => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:21:41.910519 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87a5255-3b5a-4850-8ade-c76e7b124c21 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:42.005226 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c0e9ff-0374-400e-8ee5-fbc8491a4b0c None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:42.051230 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294d86ba-580e-4a76-b480-881dd76e3885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:42.052244 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294d86ba-580e-4a76-b480-881dd76e3885 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:42.053213 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294d86ba-580e-4a76-b480-881dd76e3885 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:42.053788 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294d86ba-580e-4a76-b480-881dd76e3885 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:42.068500 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294d86ba-580e-4a76-b480-881dd76e3885 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:21:42.074678 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-294d86ba-580e-4a76-b480-881dd76e3885 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 06:21:42.074678 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:21:42.074678 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:21:42.074678 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:21:42.074678 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:21:42.074678 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:21:42.074678 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:21:42.074678 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:21:42.074678 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:21:42.074678 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:21:42.074678 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:21:42.074678 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:21:42.074678 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:21:42.074678 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:21:42.074678 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:21:42.074678 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:21:42.076142 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:21:42.076142 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:21:42.076142 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:21:42.076142 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:21:42.076142 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:21:42.076142 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:21:42.076142 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:21:42.076142 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:21:42.076142 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:21:42.076142 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:21:42.076142 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:21:42.076142 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:21:42.076142 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:21:42.076142 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:21:42.076142 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:21:42.076142 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:21:42.076937 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:21:42.076937 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:21:42.076937 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:21:42.076937 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:21:42.076937 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:21:42.076937 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:21:42.076937 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:21:42.076937 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:21:42.076937 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:21:42.076937 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:21:42.076937 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 06:21:42.076937 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:21:42.079635 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 195/393] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 24 06:21:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:21:42.086316 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294d86ba-580e-4a76-b480-881dd76e3885 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:42.156505 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87a5255-3b5a-4850-8ade-c76e7b124c21 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:42.192672 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:42.202641 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfa8614-bb76-45b5-b141-4546b26d2d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:42.203401 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfa8614-bb76-45b5-b141-4546b26d2d4b None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:42.203796 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfa8614-bb76-45b5-b141-4546b26d2d4b None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:42.204169 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfa8614-bb76-45b5-b141-4546b26d2d4b None admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:42.204971 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfa8614-bb76-45b5-b141-4546b26d2d4b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:42.205919 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfa8614-bb76-45b5-b141-4546b26d2d4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:42.206890 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfa8614-bb76-45b5-b141-4546b26d2d4b None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:42.217377 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 199/394] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 24 06:21:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:42.219024 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ef70cf-9b8a-47bc-b06f-2eb985f67e27 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:42.219292 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ef70cf-9b8a-47bc-b06f-2eb985f67e27 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:42.219567 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ef70cf-9b8a-47bc-b06f-2eb985f67e27 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:42.220800 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 200/395] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:21:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:21:42.224015 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ef70cf-9b8a-47bc-b06f-2eb985f67e27 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:42.245098 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294d86ba-580e-4a76-b480-881dd76e3885 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:42.271560 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:42.295416 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073bd844-7c62-43e6-a55d-8eb88a8608c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:42.296182 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073bd844-7c62-43e6-a55d-8eb88a8608c0 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:42.296552 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073bd844-7c62-43e6-a55d-8eb88a8608c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:42.296938 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073bd844-7c62-43e6-a55d-8eb88a8608c0 None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:42.297791 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073bd844-7c62-43e6-a55d-8eb88a8608c0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:42.298211 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073bd844-7c62-43e6-a55d-8eb88a8608c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:42.299851 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073bd844-7c62-43e6-a55d-8eb88a8608c0 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:42.311692 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 196/396] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Aug 24 06:21:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:42.317493 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36be4709-a837-4c8f-aaa4-addf01712d76 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:42.317853 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36be4709-a837-4c8f-aaa4-addf01712d76 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:42.318291 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36be4709-a837-4c8f-aaa4-addf01712d76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:42.337431 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-36be4709-a837-4c8f-aaa4-addf01712d76 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:21:42.338757 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-36be4709-a837-4c8f-aaa4-addf01712d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:21:42.366543 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ef70cf-9b8a-47bc-b06f-2eb985f67e27 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:42.413080 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8467f86-ef98-4760-80e8-b634158c78c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:42.414172 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8467f86-ef98-4760-80e8-b634158c78c6 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:42.414682 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8467f86-ef98-4760-80e8-b634158c78c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:42.415092 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8467f86-ef98-4760-80e8-b634158c78c6 None admin] PATH_INFO: `/v3/groups/9fb7ee19cac249789e1821689a52489e` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:42.416360 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8467f86-ef98-4760-80e8-b634158c78c6 None admin] RBAC: Authorizing `identity:get_group(group_id=9fb7ee19cac249789e1821689a52489e)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:42.416484 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 197/397] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:21:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:21:42.417430 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36be4709-a837-4c8f-aaa4-addf01712d76 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:42.418435 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8467f86-ef98-4760-80e8-b634158c78c6 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:42.421356 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 201/398] 10.222.0.47 () {56 vars in 1180 bytes} [Wed Aug 24 06:21:42 2022] GET /identity/v3/groups/9fb7ee19cac249789e1821689a52489e => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:21:42.433311 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8467f86-ef98-4760-80e8-b634158c78c6 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:42.526810 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:42.580196 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36be4709-a837-4c8f-aaa4-addf01712d76 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:42.627153 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8467f86-ef98-4760-80e8-b634158c78c6 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:42.629560 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd25ba87-bc9e-45bb-8721-85d771aeaa68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:42.630282 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:42.631415 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd25ba87-bc9e-45bb-8721-85d771aeaa68 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:42.631925 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd25ba87-bc9e-45bb-8721-85d771aeaa68 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:42.632401 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd25ba87-bc9e-45bb-8721-85d771aeaa68 None admin] PATH_INFO: `/v3/groups/9fb7ee19cac249789e1821689a52489e` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:42.633809 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd25ba87-bc9e-45bb-8721-85d771aeaa68 None admin] RBAC: Authorizing `identity:get_group(group_id=9fb7ee19cac249789e1821689a52489e)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:42.635487 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd25ba87-bc9e-45bb-8721-85d771aeaa68 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:42.637375 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 198/399] 10.222.0.47 () {56 vars in 1180 bytes} [Wed Aug 24 06:21:42 2022] GET /identity/v3/groups/9fb7ee19cac249789e1821689a52489e => generated 226 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:42.639510 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f006e6a-4c3c-428f-96b1-a0d74cf2fcf1 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:42.640035 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f006e6a-4c3c-428f-96b1-a0d74cf2fcf1 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:42.640475 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f006e6a-4c3c-428f-96b1-a0d74cf2fcf1 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:42.642122 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 199/400] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:21:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:21:42.647003 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f006e6a-4c3c-428f-96b1-a0d74cf2fcf1 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:42.674527 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474ab2c8-96c1-4066-89fa-e2df1849d67b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:42.675230 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ab2c8-96c1-4066-89fa-e2df1849d67b None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:42.675441 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ab2c8-96c1-4066-89fa-e2df1849d67b None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:42.675699 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ab2c8-96c1-4066-89fa-e2df1849d67b None admin] PATH_INFO: `/v3/projects/df345274c3b840b486c303c6b301052e` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:42.676815 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474ab2c8-96c1-4066-89fa-e2df1849d67b None admin] RBAC: Authorizing `identity:get_project(project_id=df345274c3b840b486c303c6b301052e)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:42.678284 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474ab2c8-96c1-4066-89fa-e2df1849d67b None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:42.680243 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 202/401] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 24 06:21:42 2022] GET /identity/v3/projects/df345274c3b840b486c303c6b301052e => generated 298 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:42.681450 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae21a86-c5f7-43ec-a980-e870fe06a02a None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:42.681733 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae21a86-c5f7-43ec-a980-e870fe06a02a None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:42.682007 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae21a86-c5f7-43ec-a980-e870fe06a02a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:42.701023 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-6ae21a86-c5f7-43ec-a980-e870fe06a02a None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:21:42.701951 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-6ae21a86-c5f7-43ec-a980-e870fe06a02a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:21:42.764054 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f006e6a-4c3c-428f-96b1-a0d74cf2fcf1 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:42.774541 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 203/402] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:21:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:21:42.774541 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae21a86-c5f7-43ec-a980-e870fe06a02a None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:42.799835 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:42.815749 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f304f19-bb63-4807-9b45-b31419ac98f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:42.816672 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f304f19-bb63-4807-9b45-b31419ac98f4 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:42.817143 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f304f19-bb63-4807-9b45-b31419ac98f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:42.817659 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f304f19-bb63-4807-9b45-b31419ac98f4 None admin] PATH_INFO: `/v3/projects/c2a0c48ef166451b912daa9e82ddcd09` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:42.819476 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f304f19-bb63-4807-9b45-b31419ac98f4 None admin] RBAC: Authorizing `identity:get_project(project_id=c2a0c48ef166451b912daa9e82ddcd09)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:42.821683 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f304f19-bb63-4807-9b45-b31419ac98f4 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:42.824479 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 200/403] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 24 06:21:42 2022] GET /identity/v3/projects/c2a0c48ef166451b912daa9e82ddcd09 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:42.825867 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f304f19-bb63-4807-9b45-b31419ac98f4 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:42.894502 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae21a86-c5f7-43ec-a980-e870fe06a02a None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:42.896062 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:42.951331 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fa15f1-02aa-4b61-8c20-61e962c98634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:42.952648 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fa15f1-02aa-4b61-8c20-61e962c98634 None admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:42.953136 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fa15f1-02aa-4b61-8c20-61e962c98634 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:42.953702 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fa15f1-02aa-4b61-8c20-61e962c98634 None admin] PATH_INFO: `/v3/projects/df345274c3b840b486c303c6b301052e/groups/9fb7ee19cac249789e1821689a52489e/roles/b71c87e12dc1452ea3c99a7bf28585ca` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:42.956212 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f304f19-bb63-4807-9b45-b31419ac98f4 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:42.957213 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fa15f1-02aa-4b61-8c20-61e962c98634 None admin] RBAC: Authorizing `identity:create_grant(project_id=df345274c3b840b486c303c6b301052e, group_id=9fb7ee19cac249789e1821689a52489e, role_id=b71c87e12dc1452ea3c99a7bf28585ca)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:42.959996 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fa15f1-02aa-4b61-8c20-61e962c98634 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:43.002147 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6f7ce6-62a1-4563-af5e-6dd2cfbb79ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:43.003569 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6f7ce6-62a1-4563-af5e-6dd2cfbb79ca None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:43.004237 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6f7ce6-62a1-4563-af5e-6dd2cfbb79ca None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:43.004902 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6f7ce6-62a1-4563-af5e-6dd2cfbb79ca None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:43.017461 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6f7ce6-62a1-4563-af5e-6dd2cfbb79ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:21:43.023389 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-bf6f7ce6-62a1-4563-af5e-6dd2cfbb79ca None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 06:21:43.023389 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:21:43.023389 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:21:43.023389 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:21:43.023389 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:21:43.023389 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:21:43.023389 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:21:43.023389 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:21:43.023389 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:21:43.023389 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:21:43.023389 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:21:43.023389 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:21:43.023389 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:21:43.023389 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:21:43.023389 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:21:43.023389 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:21:43.024297 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:21:43.024297 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:21:43.024297 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:21:43.024297 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:21:43.024297 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:21:43.024297 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:21:43.024297 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:21:43.024297 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:21:43.024297 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:21:43.024297 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:21:43.024297 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:21:43.024297 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:21:43.024297 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:21:43.024297 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:21:43.024297 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:21:43.024297 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:21:43.025079 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:21:43.025079 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:21:43.025079 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:21:43.025079 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:21:43.025079 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:21:43.025079 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:21:43.025079 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:21:43.025079 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:21:43.025079 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:21:43.025079 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:21:43.025079 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 06:21:43.025079 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:21:43.028501 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 201/404] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Aug 24 06:21:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:21:43.029707 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6f7ce6-62a1-4563-af5e-6dd2cfbb79ca None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:43.042032 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 204/405] 10.222.0.47 () {58 vars in 1442 bytes} [Wed Aug 24 06:21:42 2022] PUT /identity/v3/projects/df345274c3b840b486c303c6b301052e/groups/9fb7ee19cac249789e1821689a52489e/roles/b71c87e12dc1452ea3c99a7bf28585ca => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:21:43.050821 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fa15f1-02aa-4b61-8c20-61e962c98634 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:43.110870 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:43.191832 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6f7ce6-62a1-4563-af5e-6dd2cfbb79ca None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:43.230038 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fa15f1-02aa-4b61-8c20-61e962c98634 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:43.251633 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58362658-b67b-4086-ab10-4d680d4bc5a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:43.252321 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58362658-b67b-4086-ab10-4d680d4bc5a7 None admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:43.252885 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58362658-b67b-4086-ab10-4d680d4bc5a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:43.253267 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58362658-b67b-4086-ab10-4d680d4bc5a7 None admin] PATH_INFO: `/v3/projects/c2a0c48ef166451b912daa9e82ddcd09/groups/9fb7ee19cac249789e1821689a52489e/roles/e1f9b167c51c4a388cbc9594db8a465d` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:43.255162 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58362658-b67b-4086-ab10-4d680d4bc5a7 None admin] RBAC: Authorizing `identity:create_grant(project_id=c2a0c48ef166451b912daa9e82ddcd09, group_id=9fb7ee19cac249789e1821689a52489e, role_id=e1f9b167c51c4a388cbc9594db8a465d)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:43.257138 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58362658-b67b-4086-ab10-4d680d4bc5a7 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:43.271501 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b87004e-7563-4499-9ba0-ad40f0e6d147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:43.275682 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b87004e-7563-4499-9ba0-ad40f0e6d147 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:43.275682 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b87004e-7563-4499-9ba0-ad40f0e6d147 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:43.275682 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b87004e-7563-4499-9ba0-ad40f0e6d147 None admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:43.275682 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b87004e-7563-4499-9ba0-ad40f0e6d147 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:43.275682 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b87004e-7563-4499-9ba0-ad40f0e6d147 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:43.275682 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b87004e-7563-4499-9ba0-ad40f0e6d147 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:43.285276 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 205/406] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 24 06:21:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:21:43.285966 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b87004e-7563-4499-9ba0-ad40f0e6d147 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:43.314026 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:43.360306 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 202/407] 10.222.0.47 () {58 vars in 1442 bytes} [Wed Aug 24 06:21:43 2022] PUT /identity/v3/projects/c2a0c48ef166451b912daa9e82ddcd09/groups/9fb7ee19cac249789e1821689a52489e/roles/e1f9b167c51c4a388cbc9594db8a465d => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:21:43.361965 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58362658-b67b-4086-ab10-4d680d4bc5a7 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:43.451589 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b87004e-7563-4499-9ba0-ad40f0e6d147 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:43.537488 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6c8e7d-45a0-4dbe-a08f-cba9f1b1a1d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:43.538226 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6c8e7d-45a0-4dbe-a08f-cba9f1b1a1d9 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:43.538554 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6c8e7d-45a0-4dbe-a08f-cba9f1b1a1d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:43.538827 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6c8e7d-45a0-4dbe-a08f-cba9f1b1a1d9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:43.551094 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6c8e7d-45a0-4dbe-a08f-cba9f1b1a1d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:21:43.556379 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-4a6c8e7d-45a0-4dbe-a08f-cba9f1b1a1d9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:21:43.556379 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:21:43.556379 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:21:43.556379 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:21:43.556379 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:21:43.556379 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:21:43.556379 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:21:43.556379 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:21:43.556379 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:21:43.556379 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:21:43.556379 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:21:43.556379 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:21:43.556379 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:21:43.556379 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:21:43.556379 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:21:43.556379 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:21:43.558115 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:21:43.558115 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:21:43.558115 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:21:43.558115 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:21:43.558115 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:21:43.558115 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:21:43.558115 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:21:43.558115 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:21:43.558115 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:21:43.558115 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:21:43.558115 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:21:43.558115 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:21:43.558115 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:21:43.558115 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:21:43.558115 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:21:43.558115 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:21:43.559723 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:21:43.559723 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:21:43.559723 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:21:43.559723 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:21:43.559723 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:21:43.559723 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:21:43.559723 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:21:43.559723 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:21:43.559723 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:21:43.559723 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:21:43.559723 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:21:43.559723 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:21:43.559723 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 206/408] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Aug 24 06:21:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:21:43.564311 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6c8e7d-45a0-4dbe-a08f-cba9f1b1a1d9 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:43.565337 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58362658-b67b-4086-ab10-4d680d4bc5a7 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:43.582673 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:43.614729 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd74d4c2-1d77-46ce-845a-5dacb784aeff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:43.615395 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd74d4c2-1d77-46ce-845a-5dacb784aeff None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:43.615674 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd74d4c2-1d77-46ce-845a-5dacb784aeff None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:43.615947 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd74d4c2-1d77-46ce-845a-5dacb784aeff None admin] PATH_INFO: `/v3/groups/9fb7ee19cac249789e1821689a52489e` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:43.618652 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd74d4c2-1d77-46ce-845a-5dacb784aeff None admin] RBAC: Authorizing `identity:get_group(group_id=9fb7ee19cac249789e1821689a52489e)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:43.620122 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd74d4c2-1d77-46ce-845a-5dacb784aeff None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:43.627245 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 203/409] 10.222.0.47 () {56 vars in 1180 bytes} [Wed Aug 24 06:21:43 2022] GET /identity/v3/groups/9fb7ee19cac249789e1821689a52489e => generated 226 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:43.634978 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd74d4c2-1d77-46ce-845a-5dacb784aeff None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:43.748363 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:43.769336 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6c8e7d-45a0-4dbe-a08f-cba9f1b1a1d9 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:43.815328 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd74d4c2-1d77-46ce-845a-5dacb784aeff None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:43.818804 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58e98b2-7d1f-4fed-a3dc-9d53fc4c4f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:43.819734 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58e98b2-7d1f-4fed-a3dc-9d53fc4c4f14 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:43.820233 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58e98b2-7d1f-4fed-a3dc-9d53fc4c4f14 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:43.820715 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58e98b2-7d1f-4fed-a3dc-9d53fc4c4f14 None admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:43.821631 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58e98b2-7d1f-4fed-a3dc-9d53fc4c4f14 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:43.822129 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58e98b2-7d1f-4fed-a3dc-9d53fc4c4f14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:43.829450 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58e98b2-7d1f-4fed-a3dc-9d53fc4c4f14 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:43.844132 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 207/410] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 24 06:21:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:43.845932 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58e98b2-7d1f-4fed-a3dc-9d53fc4c4f14 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:43.852050 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a061f0-1547-47c5-a9b2-4d20b2f0f162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:43.852699 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a061f0-1547-47c5-a9b2-4d20b2f0f162 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:43.852987 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a061f0-1547-47c5-a9b2-4d20b2f0f162 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:43.853265 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a061f0-1547-47c5-a9b2-4d20b2f0f162 None admin] PATH_INFO: `/v3/projects/c2a0c48ef166451b912daa9e82ddcd09` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:43.854496 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a061f0-1547-47c5-a9b2-4d20b2f0f162 None admin] RBAC: Authorizing `identity:get_project(project_id=c2a0c48ef166451b912daa9e82ddcd09)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:43.855934 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a061f0-1547-47c5-a9b2-4d20b2f0f162 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:43.858649 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 204/411] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 24 06:21:43 2022] GET /identity/v3/projects/c2a0c48ef166451b912daa9e82ddcd09 => generated 302 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:21:43.864296 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a061f0-1547-47c5-a9b2-4d20b2f0f162 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:43.922787 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:44.005211 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a061f0-1547-47c5-a9b2-4d20b2f0f162 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:44.057474 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58e98b2-7d1f-4fed-a3dc-9d53fc4c4f14 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:44.079864 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba61a70-d45c-4951-9921-b122c6acf750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:44.080548 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba61a70-d45c-4951-9921-b122c6acf750 None admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:44.080828 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba61a70-d45c-4951-9921-b122c6acf750 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:44.081101 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba61a70-d45c-4951-9921-b122c6acf750 None admin] PATH_INFO: `/v3/projects/c2a0c48ef166451b912daa9e82ddcd09/groups/9fb7ee19cac249789e1821689a52489e/roles/b71c87e12dc1452ea3c99a7bf28585ca` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:44.087187 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba61a70-d45c-4951-9921-b122c6acf750 None admin] RBAC: Authorizing `identity:create_grant(project_id=c2a0c48ef166451b912daa9e82ddcd09, group_id=9fb7ee19cac249789e1821689a52489e, role_id=b71c87e12dc1452ea3c99a7bf28585ca)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:44.088870 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba61a70-d45c-4951-9921-b122c6acf750 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:44.097943 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de541846-a587-4554-abba-753d4f53354a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:44.098686 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de541846-a587-4554-abba-753d4f53354a None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:44.098970 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de541846-a587-4554-abba-753d4f53354a None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:44.099242 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de541846-a587-4554-abba-753d4f53354a None admin] PATH_INFO: `/v3/groups/2faefd426a0748d6a8263d0eb0102204` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:44.100495 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de541846-a587-4554-abba-753d4f53354a None admin] RBAC: Authorizing `identity:get_group(group_id=2faefd426a0748d6a8263d0eb0102204)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:44.102086 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de541846-a587-4554-abba-753d4f53354a None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:44.104114 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 208/412] 10.222.0.47 () {56 vars in 1180 bytes} [Wed Aug 24 06:21:43 2022] GET /identity/v3/groups/2faefd426a0748d6a8263d0eb0102204 => generated 229 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:44.110705 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de541846-a587-4554-abba-753d4f53354a None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:44.138403 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:44.181931 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 205/413] 10.222.0.47 () {58 vars in 1442 bytes} [Wed Aug 24 06:21:43 2022] PUT /identity/v3/projects/c2a0c48ef166451b912daa9e82ddcd09/groups/9fb7ee19cac249789e1821689a52489e/roles/b71c87e12dc1452ea3c99a7bf28585ca => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:21:44.184879 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ace7b26-e2ca-48b0-a535-bd6cd6de99f2 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:44.185825 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ace7b26-e2ca-48b0-a535-bd6cd6de99f2 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:44.186170 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ace7b26-e2ca-48b0-a535-bd6cd6de99f2 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:44.190542 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 206/414] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:21:44 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:21:44.196845 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85e3408-d606-45f4-b071-a22d71f35070 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:44.197135 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85e3408-d606-45f4-b071-a22d71f35070 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:44.200576 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85e3408-d606-45f4-b071-a22d71f35070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:44.240852 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-d85e3408-d606-45f4-b071-a22d71f35070 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:21:44.242140 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-d85e3408-d606-45f4-b071-a22d71f35070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:21:44.298326 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de541846-a587-4554-abba-753d4f53354a None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:44.368653 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c084cdaa-dc6b-4e4e-a53a-2a5c25ee6c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:44.371440 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c084cdaa-dc6b-4e4e-a53a-2a5c25ee6c06 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:44.371440 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c084cdaa-dc6b-4e4e-a53a-2a5c25ee6c06 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:44.371440 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c084cdaa-dc6b-4e4e-a53a-2a5c25ee6c06 None admin] PATH_INFO: `/v3/projects/24c1e9e70e07496595a602ca047e6a1f` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:44.374270 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c084cdaa-dc6b-4e4e-a53a-2a5c25ee6c06 None admin] RBAC: Authorizing `identity:get_project(project_id=24c1e9e70e07496595a602ca047e6a1f)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:44.376889 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c084cdaa-dc6b-4e4e-a53a-2a5c25ee6c06 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:44.378339 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 209/415] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 24 06:21:44 2022] GET /identity/v3/projects/24c1e9e70e07496595a602ca047e6a1f => generated 344 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:44.385266 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c084cdaa-dc6b-4e4e-a53a-2a5c25ee6c06 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:44.409167 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 207/416] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:21:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:21:44.486022 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:44.505737 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c084cdaa-dc6b-4e4e-a53a-2a5c25ee6c06 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:44.564814 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbcbc10-5ecf-4d1f-94ff-55995ff22ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:44.565442 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbcbc10-5ecf-4d1f-94ff-55995ff22ec2 None admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:44.565725 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbcbc10-5ecf-4d1f-94ff-55995ff22ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:44.566005 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbcbc10-5ecf-4d1f-94ff-55995ff22ec2 None admin] PATH_INFO: `/v3/projects/24c1e9e70e07496595a602ca047e6a1f/groups/2faefd426a0748d6a8263d0eb0102204/roles/486144ad1bcd417e86acdbc1d14327bf` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:44.567979 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbcbc10-5ecf-4d1f-94ff-55995ff22ec2 None admin] RBAC: Authorizing `identity:create_grant(project_id=24c1e9e70e07496595a602ca047e6a1f, group_id=2faefd426a0748d6a8263d0eb0102204, role_id=486144ad1bcd417e86acdbc1d14327bf)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:44.569467 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbcbc10-5ecf-4d1f-94ff-55995ff22ec2 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:44.640560 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 210/417] 10.222.0.47 () {58 vars in 1442 bytes} [Wed Aug 24 06:21:44 2022] PUT /identity/v3/projects/24c1e9e70e07496595a602ca047e6a1f/groups/2faefd426a0748d6a8263d0eb0102204/roles/486144ad1bcd417e86acdbc1d14327bf => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:21:44.926056 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85e3408-d606-45f4-b071-a22d71f35070 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:44.978847 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:45.055149 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85e3408-d606-45f4-b071-a22d71f35070 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:45.105222 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fb997d-10c1-4c09-a0c2-350a8e97d9a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:45.106279 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fb997d-10c1-4c09-a0c2-350a8e97d9a4 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:45.106755 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fb997d-10c1-4c09-a0c2-350a8e97d9a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:45.107220 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fb997d-10c1-4c09-a0c2-350a8e97d9a4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:45.124346 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fb997d-10c1-4c09-a0c2-350a8e97d9a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:21:45.129507 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-81fb997d-10c1-4c09-a0c2-350a8e97d9a4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 06:21:45.129507 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:21:45.129507 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:21:45.129507 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:21:45.129507 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:21:45.129507 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:21:45.129507 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:21:45.129507 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:21:45.129507 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:21:45.129507 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:21:45.129507 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:21:45.129507 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:21:45.129507 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:21:45.129507 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:21:45.129507 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:21:45.129507 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:21:45.130376 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:21:45.130376 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:21:45.130376 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:21:45.130376 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:21:45.130376 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:21:45.130376 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:21:45.130376 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:21:45.130376 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:21:45.130376 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:21:45.130376 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:21:45.130376 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:21:45.130376 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:21:45.130376 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:21:45.130376 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:21:45.130376 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:21:45.130376 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:21:45.131214 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:21:45.131214 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:21:45.131214 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:21:45.131214 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:21:45.131214 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:21:45.131214 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:21:45.131214 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:21:45.131214 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:21:45.131214 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:21:45.131214 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:21:45.131214 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 06:21:45.131214 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:21:45.133528 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 208/418] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 24 06:21:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:21:45.140190 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbcbc10-5ecf-4d1f-94ff-55995ff22ec2 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:45.285204 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b41bdc-8673-4a49-a679-2d6a2b92afb7 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:45.285499 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b41bdc-8673-4a49-a679-2d6a2b92afb7 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:45.285725 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b41bdc-8673-4a49-a679-2d6a2b92afb7 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:45.287314 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 209/419] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:21:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 06:21:45.300661 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbcbc10-5ecf-4d1f-94ff-55995ff22ec2 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:45.312825 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42db1d7-8717-4ec6-b317-dd05fc29b3a0 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:45.313133 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42db1d7-8717-4ec6-b317-dd05fc29b3a0 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:45.313409 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42db1d7-8717-4ec6-b317-dd05fc29b3a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:45.349524 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-e42db1d7-8717-4ec6-b317-dd05fc29b3a0 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:21:45.352481 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-e42db1d7-8717-4ec6-b317-dd05fc29b3a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:21:45.361363 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa24dde-da34-4418-9ead-2c0a61b1beb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:45.362356 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa24dde-da34-4418-9ead-2c0a61b1beb5 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:45.362839 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa24dde-da34-4418-9ead-2c0a61b1beb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:45.363341 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa24dde-da34-4418-9ead-2c0a61b1beb5 None admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:45.366546 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa24dde-da34-4418-9ead-2c0a61b1beb5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:45.367087 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa24dde-da34-4418-9ead-2c0a61b1beb5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:45.368729 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa24dde-da34-4418-9ead-2c0a61b1beb5 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:45.387929 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 211/420] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Aug 24 06:21:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:45.394253 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa24dde-da34-4418-9ead-2c0a61b1beb5 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:45.400898 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:45.490173 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 210/421] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:21:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 06:21:45.566605 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa24dde-da34-4418-9ead-2c0a61b1beb5 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:45.619016 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6837c3a-ef92-418b-a6b6-ed2863dfbe23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:45.619687 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6837c3a-ef92-418b-a6b6-ed2863dfbe23 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:45.619977 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6837c3a-ef92-418b-a6b6-ed2863dfbe23 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:45.620248 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6837c3a-ef92-418b-a6b6-ed2863dfbe23 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:45.707112 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6837c3a-ef92-418b-a6b6-ed2863dfbe23 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:45.708582 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6837c3a-ef92-418b-a6b6-ed2863dfbe23 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:45.711511 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:45.831914 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 212/422] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 24 06:21:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 06:21:45.833900 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42db1d7-8717-4ec6-b317-dd05fc29b3a0 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:45.837337 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6837c3a-ef92-418b-a6b6-ed2863dfbe23 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:45.874380 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:45.975308 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42db1d7-8717-4ec6-b317-dd05fc29b3a0 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:45.976213 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6837c3a-ef92-418b-a6b6-ed2863dfbe23 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:45.980213 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:46.019501 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15e4762-42a7-4f0b-8bb7-b0dcee1820fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:46.020438 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15e4762-42a7-4f0b-8bb7-b0dcee1820fe None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:46.020884 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15e4762-42a7-4f0b-8bb7-b0dcee1820fe None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:46.021329 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15e4762-42a7-4f0b-8bb7-b0dcee1820fe None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:46.025474 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb7d32d-e596-4f86-ae13-2116d354223b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:46.026531 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb7d32d-e596-4f86-ae13-2116d354223b None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:46.027018 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb7d32d-e596-4f86-ae13-2116d354223b None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:46.027479 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb7d32d-e596-4f86-ae13-2116d354223b None admin] PATH_INFO: `/v3/projects/df345274c3b840b486c303c6b301052e` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:46.029245 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb7d32d-e596-4f86-ae13-2116d354223b None admin] RBAC: Authorizing `identity:get_project(project_id=df345274c3b840b486c303c6b301052e)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:46.030996 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb7d32d-e596-4f86-ae13-2116d354223b None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:46.033451 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 213/423] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 24 06:21:45 2022] GET /identity/v3/projects/df345274c3b840b486c303c6b301052e => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:21:46.034976 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425b3625-63c7-4bec-867d-c9655b3840ae None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:46.035669 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425b3625-63c7-4bec-867d-c9655b3840ae None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:46.036331 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425b3625-63c7-4bec-867d-c9655b3840ae None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:46.038267 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 214/424] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:21:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:21:46.039967 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425b3625-63c7-4bec-867d-c9655b3840ae None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:46.040671 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15e4762-42a7-4f0b-8bb7-b0dcee1820fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:21:46.049844 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-d15e4762-42a7-4f0b-8bb7-b0dcee1820fe None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 06:21:46.049844 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:21:46.049844 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:21:46.049844 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:21:46.049844 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:21:46.049844 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:21:46.049844 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:21:46.049844 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:21:46.049844 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:21:46.049844 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:21:46.049844 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:21:46.049844 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:21:46.049844 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:21:46.049844 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:21:46.049844 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:21:46.049844 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:21:46.050964 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:21:46.050964 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:21:46.050964 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:21:46.050964 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:21:46.050964 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:21:46.050964 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:21:46.050964 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:21:46.050964 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:21:46.050964 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:21:46.050964 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:21:46.050964 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:21:46.050964 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:21:46.050964 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:21:46.050964 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:21:46.050964 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:21:46.050964 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:21:46.051825 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:21:46.051825 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:21:46.051825 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:21:46.051825 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:21:46.051825 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:21:46.051825 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:21:46.051825 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:21:46.051825 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:21:46.051825 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:21:46.051825 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:21:46.051825 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 06:21:46.051825 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:21:46.051825 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 211/425] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Aug 24 06:21:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:21:46.058449 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492afd36-87f8-4b5e-8438-fc86b3c39f91 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:46.058449 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492afd36-87f8-4b5e-8438-fc86b3c39f91 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:46.058449 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492afd36-87f8-4b5e-8438-fc86b3c39f91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:46.085204 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-492afd36-87f8-4b5e-8438-fc86b3c39f91 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:21:46.087597 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-492afd36-87f8-4b5e-8438-fc86b3c39f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:21:46.167379 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425b3625-63c7-4bec-867d-c9655b3840ae None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:46.180175 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 212/426] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:21:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:21:46.181682 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492afd36-87f8-4b5e-8438-fc86b3c39f91 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:46.206940 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:46.252424 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4ac80e-a8b2-4409-898d-0adad21dfb94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:46.253322 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4ac80e-a8b2-4409-898d-0adad21dfb94 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:46.253724 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4ac80e-a8b2-4409-898d-0adad21dfb94 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:46.254114 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4ac80e-a8b2-4409-898d-0adad21dfb94 None admin] PATH_INFO: `/v3/groups/9fb7ee19cac249789e1821689a52489e` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:46.257925 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4ac80e-a8b2-4409-898d-0adad21dfb94 None admin] RBAC: Authorizing `identity:get_group(group_id=9fb7ee19cac249789e1821689a52489e)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:46.259539 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4ac80e-a8b2-4409-898d-0adad21dfb94 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:46.263246 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 215/427] 10.222.0.47 () {56 vars in 1180 bytes} [Wed Aug 24 06:21:46 2022] GET /identity/v3/groups/9fb7ee19cac249789e1821689a52489e => generated 226 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:46.268549 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4ac80e-a8b2-4409-898d-0adad21dfb94 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:46.313343 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492afd36-87f8-4b5e-8438-fc86b3c39f91 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:46.364239 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa941170-3613-41c9-8b0a-d8dd27c9f3ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:46.365100 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa941170-3613-41c9-8b0a-d8dd27c9f3ad None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:46.365539 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa941170-3613-41c9-8b0a-d8dd27c9f3ad None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:46.365921 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:46.366413 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa941170-3613-41c9-8b0a-d8dd27c9f3ad None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:46.367632 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa941170-3613-41c9-8b0a-d8dd27c9f3ad None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:46.368079 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa941170-3613-41c9-8b0a-d8dd27c9f3ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:46.370249 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa941170-3613-41c9-8b0a-d8dd27c9f3ad None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:46.381510 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 213/428] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 24 06:21:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:46.387772 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa941170-3613-41c9-8b0a-d8dd27c9f3ad None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:46.486743 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:46.488147 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4ac80e-a8b2-4409-898d-0adad21dfb94 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:46.510711 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa941170-3613-41c9-8b0a-d8dd27c9f3ad None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:46.553155 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ffa2d3-fe4e-48f7-adac-c71385bef69f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:46.554449 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ffa2d3-fe4e-48f7-adac-c71385bef69f None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:46.554960 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ffa2d3-fe4e-48f7-adac-c71385bef69f None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:46.555422 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ffa2d3-fe4e-48f7-adac-c71385bef69f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:46.582766 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5138ffe-df16-4e1a-9653-4cbf375b0440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:46.583397 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5138ffe-df16-4e1a-9653-4cbf375b0440 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:46.583681 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5138ffe-df16-4e1a-9653-4cbf375b0440 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:46.583957 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5138ffe-df16-4e1a-9653-4cbf375b0440 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:46.584861 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5138ffe-df16-4e1a-9653-4cbf375b0440 None admin] RBAC: Adding query filter params (group.id=9fb7ee19cac249789e1821689a52489e, role.id=e1f9b167c51c4a388cbc9594db8a465d, scope.project.id=df345274c3b840b486c303c6b301052e) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:46.585345 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5138ffe-df16-4e1a-9653-4cbf375b0440 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:46.586951 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5138ffe-df16-4e1a-9653-4cbf375b0440 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:46.608112 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 216/429] 10.222.0.47 () {56 vars in 1376 bytes} [Wed Aug 24 06:21:46 2022] GET /identity/v3/role_assignments?group.id=9fb7ee19cac249789e1821689a52489e&scope.project.id=df345274c3b840b486c303c6b301052e&role.id=e1f9b167c51c4a388cbc9594db8a465d => generated 611 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:21:46.639550 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ffa2d3-fe4e-48f7-adac-c71385bef69f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:46.644437 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ffa2d3-fe4e-48f7-adac-c71385bef69f None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:46.648143 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2609292f-11d5-41c5-9ced-878f4d93c2e1 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:46.648660 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2609292f-11d5-41c5-9ced-878f4d93c2e1 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:46.649109 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2609292f-11d5-41c5-9ced-878f4d93c2e1 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:46.657330 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 217/430] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:21:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 06:21:46.659219 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a64779-79e5-4876-aef5-650a62d22dcf None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:46.659718 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a64779-79e5-4876-aef5-650a62d22dcf None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:46.660194 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a64779-79e5-4876-aef5-650a62d22dcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:46.687296 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-43a64779-79e5-4876-aef5-650a62d22dcf None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:21:46.688863 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-43a64779-79e5-4876-aef5-650a62d22dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:21:46.746312 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:46.773219 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 214/431] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 24 06:21:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 06:21:46.773219 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ffa2d3-fe4e-48f7-adac-c71385bef69f None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:46.788897 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 218/432] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:21:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 06:21:46.789825 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a64779-79e5-4876-aef5-650a62d22dcf None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:46.829377 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:46.913058 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ffa2d3-fe4e-48f7-adac-c71385bef69f None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:46.925195 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a64779-79e5-4876-aef5-650a62d22dcf None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:46.968316 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7d516e-4500-4f35-97b5-797003f25fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:46.969449 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7d516e-4500-4f35-97b5-797003f25fe0 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:46.969724 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7d516e-4500-4f35-97b5-797003f25fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:46.969991 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7d516e-4500-4f35-97b5-797003f25fe0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:46.975589 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb8b53a-e6c2-444a-be18-3786e60c7bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:46.976523 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb8b53a-e6c2-444a-be18-3786e60c7bb3 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:46.977021 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb8b53a-e6c2-444a-be18-3786e60c7bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:46.977512 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb8b53a-e6c2-444a-be18-3786e60c7bb3 None admin] PATH_INFO: `/v3/projects/df345274c3b840b486c303c6b301052e` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:46.978932 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb8b53a-e6c2-444a-be18-3786e60c7bb3 None admin] RBAC: Authorizing `identity:get_project(project_id=df345274c3b840b486c303c6b301052e)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:46.980798 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb8b53a-e6c2-444a-be18-3786e60c7bb3 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:46.984175 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 219/433] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 24 06:21:46 2022] GET /identity/v3/projects/df345274c3b840b486c303c6b301052e => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:46.988138 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7d516e-4500-4f35-97b5-797003f25fe0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:21:46.990690 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb8b53a-e6c2-444a-be18-3786e60c7bb3 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:46.994285 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-6f7d516e-4500-4f35-97b5-797003f25fe0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 06:21:46.994285 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:21:46.994285 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:21:46.994285 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:21:46.994285 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:21:46.994285 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:21:46.994285 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:21:46.994285 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:21:46.994285 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:21:46.994285 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:21:46.994285 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:21:46.994285 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:21:46.994285 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:21:46.994285 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:21:46.994285 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:21:46.994285 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:21:46.995199 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:21:46.995199 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:21:46.995199 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:21:46.995199 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:21:46.995199 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:21:46.995199 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:21:46.995199 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:21:46.995199 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:21:46.995199 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:21:46.995199 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:21:46.995199 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:21:46.995199 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:21:46.995199 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:21:46.995199 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:21:46.995199 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:21:46.995199 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:21:46.996185 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:21:46.996185 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:21:46.996185 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:21:46.996185 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:21:46.996185 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:21:46.996185 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:21:46.996185 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:21:46.996185 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:21:46.996185 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:21:46.996185 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:21:46.996185 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 06:21:46.996185 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:21:47.000178 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 215/434] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 24 06:21:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:21:47.006258 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7d516e-4500-4f35-97b5-797003f25fe0 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:47.058755 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:47.118937 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7d516e-4500-4f35-97b5-797003f25fe0 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:47.126694 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb8b53a-e6c2-444a-be18-3786e60c7bb3 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:47.128660 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:47.169369 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc82eb50-cac6-48ae-a313-95fa45a94829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:47.170765 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc82eb50-cac6-48ae-a313-95fa45a94829 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:47.171192 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc82eb50-cac6-48ae-a313-95fa45a94829 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:47.171659 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc82eb50-cac6-48ae-a313-95fa45a94829 None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:47.172495 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc82eb50-cac6-48ae-a313-95fa45a94829 None admin] RBAC: Adding query filter params (name=member) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:47.172948 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc82eb50-cac6-48ae-a313-95fa45a94829 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:47.176596 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc82eb50-cac6-48ae-a313-95fa45a94829 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:47.180204 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdca3ef-0ad8-42a8-b054-3fb0bc8f265e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:47.180852 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdca3ef-0ad8-42a8-b054-3fb0bc8f265e None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:47.181142 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdca3ef-0ad8-42a8-b054-3fb0bc8f265e None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:47.181451 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdca3ef-0ad8-42a8-b054-3fb0bc8f265e None admin] PATH_INFO: `/v3/groups/9fb7ee19cac249789e1821689a52489e` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:47.182735 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdca3ef-0ad8-42a8-b054-3fb0bc8f265e None admin] RBAC: Authorizing `identity:get_group(group_id=9fb7ee19cac249789e1821689a52489e)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:47.184901 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdca3ef-0ad8-42a8-b054-3fb0bc8f265e None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:47.186831 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 220/435] 10.222.0.47 () {56 vars in 1180 bytes} [Wed Aug 24 06:21:46 2022] GET /identity/v3/groups/9fb7ee19cac249789e1821689a52489e => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:21:47.188269 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdca3ef-0ad8-42a8-b054-3fb0bc8f265e None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:47.190625 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 216/436] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Aug 24 06:21:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:47.194780 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc82eb50-cac6-48ae-a313-95fa45a94829 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:47.312424 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdca3ef-0ad8-42a8-b054-3fb0bc8f265e None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:47.314715 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc82eb50-cac6-48ae-a313-95fa45a94829 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:47.316296 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:47.348269 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523ce947-2b42-4be1-ad3c-8f9ccf53afc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:47.349020 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523ce947-2b42-4be1-ad3c-8f9ccf53afc3 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:47.349127 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a84292-a6fd-4fbd-84d0-0caf0acfd899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:47.349349 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523ce947-2b42-4be1-ad3c-8f9ccf53afc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:47.349627 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523ce947-2b42-4be1-ad3c-8f9ccf53afc3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:47.349736 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a84292-a6fd-4fbd-84d0-0caf0acfd899 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:47.349988 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a84292-a6fd-4fbd-84d0-0caf0acfd899 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:47.350269 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a84292-a6fd-4fbd-84d0-0caf0acfd899 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:47.351008 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a84292-a6fd-4fbd-84d0-0caf0acfd899 None admin] RBAC: Adding query filter params (group.id=9fb7ee19cac249789e1821689a52489e, role.id=b71c87e12dc1452ea3c99a7bf28585ca, scope.project.id=df345274c3b840b486c303c6b301052e) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:47.351316 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a84292-a6fd-4fbd-84d0-0caf0acfd899 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:47.352909 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a84292-a6fd-4fbd-84d0-0caf0acfd899 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:47.360119 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 217/437] 10.222.0.47 () {56 vars in 1376 bytes} [Wed Aug 24 06:21:47 2022] GET /identity/v3/role_assignments?group.id=9fb7ee19cac249789e1821689a52489e&scope.project.id=df345274c3b840b486c303c6b301052e&role.id=b71c87e12dc1452ea3c99a7bf28585ca => generated 611 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:47.361038 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a84292-a6fd-4fbd-84d0-0caf0acfd899 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:47.361298 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523ce947-2b42-4be1-ad3c-8f9ccf53afc3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:21:47.366302 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-523ce947-2b42-4be1-ad3c-8f9ccf53afc3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 06:21:47.366302 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:21:47.366302 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:21:47.366302 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:21:47.366302 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:21:47.366302 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:21:47.366302 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:21:47.366302 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:21:47.366302 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:21:47.366302 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:21:47.366302 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:21:47.366302 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:21:47.366302 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:21:47.366302 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:21:47.366302 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:21:47.366302 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:21:47.367705 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:21:47.367705 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:21:47.367705 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:21:47.367705 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:21:47.367705 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:21:47.367705 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:21:47.367705 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:21:47.367705 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:21:47.367705 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:21:47.367705 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:21:47.367705 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:21:47.367705 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:21:47.367705 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:21:47.367705 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:21:47.367705 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:21:47.367705 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:21:47.368487 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:21:47.368487 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:21:47.368487 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:21:47.368487 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:21:47.368487 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:21:47.368487 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:21:47.368487 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:21:47.368487 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:21:47.368487 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:21:47.368487 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:21:47.368487 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 06:21:47.368487 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:21:47.368487 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 221/438] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Aug 24 06:21:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:21:47.369147 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8761a3-4aff-4e26-8e10-a238351a3acf None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:47.369147 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8761a3-4aff-4e26-8e10-a238351a3acf None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:47.370478 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8761a3-4aff-4e26-8e10-a238351a3acf None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:47.371039 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 222/439] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:21:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:21:47.378212 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c99b2d-62d3-4827-9a63-e8e9b485be54 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:47.378212 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c99b2d-62d3-4827-9a63-e8e9b485be54 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:47.378708 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c99b2d-62d3-4827-9a63-e8e9b485be54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:47.397985 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-10c99b2d-62d3-4827-9a63-e8e9b485be54 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:21:47.399218 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-10c99b2d-62d3-4827-9a63-e8e9b485be54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:21:47.428012 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:47.477115 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 223/440] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:21:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 06:21:47.482529 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c99b2d-62d3-4827-9a63-e8e9b485be54 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:47.491074 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a84292-a6fd-4fbd-84d0-0caf0acfd899 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:47.548085 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e2c1e7-4aa0-4b3e-85dd-5fc285827194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:47.548085 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e2c1e7-4aa0-4b3e-85dd-5fc285827194 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:47.548085 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e2c1e7-4aa0-4b3e-85dd-5fc285827194 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:47.548085 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e2c1e7-4aa0-4b3e-85dd-5fc285827194 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:47.589103 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:47.612743 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c99b2d-62d3-4827-9a63-e8e9b485be54 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:47.648506 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e2c1e7-4aa0-4b3e-85dd-5fc285827194 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:47.651108 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e2c1e7-4aa0-4b3e-85dd-5fc285827194 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:47.668773 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5617cdd7-87e0-45de-b69f-458eae852971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:47.669586 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5617cdd7-87e0-45de-b69f-458eae852971 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:47.670274 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5617cdd7-87e0-45de-b69f-458eae852971 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:47.670560 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5617cdd7-87e0-45de-b69f-458eae852971 None admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:47.671532 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5617cdd7-87e0-45de-b69f-458eae852971 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:47.671905 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5617cdd7-87e0-45de-b69f-458eae852971 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:47.673839 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5617cdd7-87e0-45de-b69f-458eae852971 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:47.687463 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 224/441] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 24 06:21:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:47.692871 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5617cdd7-87e0-45de-b69f-458eae852971 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:47.731366 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:47.779560 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 218/442] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 24 06:21:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 06:21:47.780322 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e2c1e7-4aa0-4b3e-85dd-5fc285827194 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:47.836706 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5617cdd7-87e0-45de-b69f-458eae852971 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:47.865901 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:47.879012 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86369d6d-a44b-41a2-a9c0-028e6dc897bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:47.879727 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86369d6d-a44b-41a2-a9c0-028e6dc897bd None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:47.879959 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86369d6d-a44b-41a2-a9c0-028e6dc897bd None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:47.880254 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86369d6d-a44b-41a2-a9c0-028e6dc897bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:47.907900 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e2c1e7-4aa0-4b3e-85dd-5fc285827194 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:47.944482 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a10634-c665-4892-af0b-8b3f8a271126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:47.945058 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a10634-c665-4892-af0b-8b3f8a271126 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:47.945361 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a10634-c665-4892-af0b-8b3f8a271126 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:47.945633 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a10634-c665-4892-af0b-8b3f8a271126 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:47.954915 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86369d6d-a44b-41a2-a9c0-028e6dc897bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:47.956477 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86369d6d-a44b-41a2-a9c0-028e6dc897bd None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:47.956810 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a10634-c665-4892-af0b-8b3f8a271126 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:21:47.961991 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-b2a10634-c665-4892-af0b-8b3f8a271126 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:21:47.961991 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:21:47.961991 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:21:47.961991 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:21:47.961991 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:21:47.961991 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:21:47.961991 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:21:47.961991 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:21:47.961991 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:21:47.961991 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:21:47.961991 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:21:47.961991 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:21:47.961991 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:21:47.961991 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:21:47.961991 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:21:47.961991 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:21:47.962939 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:21:47.962939 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:21:47.962939 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:21:47.962939 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:21:47.962939 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:21:47.962939 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:21:47.962939 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:21:47.962939 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:21:47.962939 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:21:47.962939 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:21:47.962939 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:21:47.962939 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:21:47.962939 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:21:47.962939 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:21:47.962939 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:21:47.962939 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:21:47.963736 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:21:47.963736 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:21:47.963736 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:21:47.963736 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:21:47.963736 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:21:47.963736 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:21:47.963736 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:21:47.963736 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:21:47.963736 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:21:47.963736 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:21:47.963736 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:21:47.963736 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:21:47.963736 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 219/443] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Aug 24 06:21:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:21:47.964415 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a10634-c665-4892-af0b-8b3f8a271126 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:47.988392 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:48.067129 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a10634-c665-4892-af0b-8b3f8a271126 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:48.069464 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 225/444] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 24 06:21:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 06:21:48.070320 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86369d6d-a44b-41a2-a9c0-028e6dc897bd None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:48.097228 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:48.106781 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef21c2a-4a0b-4df1-9350-fe92736caa2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:48.107278 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef21c2a-4a0b-4df1-9350-fe92736caa2d None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:48.107594 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef21c2a-4a0b-4df1-9350-fe92736caa2d None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:48.107863 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef21c2a-4a0b-4df1-9350-fe92736caa2d None admin] PATH_INFO: `/v3/projects/c2a0c48ef166451b912daa9e82ddcd09` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:48.109024 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef21c2a-4a0b-4df1-9350-fe92736caa2d None admin] RBAC: Authorizing `identity:get_project(project_id=c2a0c48ef166451b912daa9e82ddcd09)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:48.115799 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef21c2a-4a0b-4df1-9350-fe92736caa2d None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:48.115799 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 220/445] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 24 06:21:47 2022] GET /identity/v3/projects/c2a0c48ef166451b912daa9e82ddcd09 => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:48.115799 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef21c2a-4a0b-4df1-9350-fe92736caa2d None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:48.182794 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86369d6d-a44b-41a2-a9c0-028e6dc897bd None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:48.206610 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:48.216804 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef21c2a-4a0b-4df1-9350-fe92736caa2d None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:48.231685 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719ca787-1d53-40a0-9ee0-bffb59cbf170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:48.231685 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719ca787-1d53-40a0-9ee0-bffb59cbf170 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:48.231685 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719ca787-1d53-40a0-9ee0-bffb59cbf170 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:48.231685 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719ca787-1d53-40a0-9ee0-bffb59cbf170 None admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:48.232017 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719ca787-1d53-40a0-9ee0-bffb59cbf170 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:48.232017 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719ca787-1d53-40a0-9ee0-bffb59cbf170 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:48.232017 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719ca787-1d53-40a0-9ee0-bffb59cbf170 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:48.232017 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 226/446] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 24 06:21:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:48.232308 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719ca787-1d53-40a0-9ee0-bffb59cbf170 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:48.256246 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadbfb4f-cbd7-4563-a77b-ae15f8ce563d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:48.256877 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadbfb4f-cbd7-4563-a77b-ae15f8ce563d None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:48.257154 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadbfb4f-cbd7-4563-a77b-ae15f8ce563d None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:48.257454 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadbfb4f-cbd7-4563-a77b-ae15f8ce563d None admin] PATH_INFO: `/v3/projects/c2a0c48ef166451b912daa9e82ddcd09` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:48.258572 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadbfb4f-cbd7-4563-a77b-ae15f8ce563d None admin] RBAC: Authorizing `identity:get_project(project_id=c2a0c48ef166451b912daa9e82ddcd09)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:48.259988 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadbfb4f-cbd7-4563-a77b-ae15f8ce563d None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:48.261769 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 221/447] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 24 06:21:48 2022] GET /identity/v3/projects/c2a0c48ef166451b912daa9e82ddcd09 => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:48.267755 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadbfb4f-cbd7-4563-a77b-ae15f8ce563d None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:48.332093 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:48.338610 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719ca787-1d53-40a0-9ee0-bffb59cbf170 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:48.375039 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b98f586-169c-49db-b28a-af3d4701910d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:48.375727 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b98f586-169c-49db-b28a-af3d4701910d None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:48.375949 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b98f586-169c-49db-b28a-af3d4701910d None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:48.376249 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b98f586-169c-49db-b28a-af3d4701910d None admin] PATH_INFO: `/v3/groups/9fb7ee19cac249789e1821689a52489e` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:48.377440 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b98f586-169c-49db-b28a-af3d4701910d None admin] RBAC: Authorizing `identity:get_group(group_id=9fb7ee19cac249789e1821689a52489e)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:48.378927 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b98f586-169c-49db-b28a-af3d4701910d None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:48.379056 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadbfb4f-cbd7-4563-a77b-ae15f8ce563d None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:48.380570 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 227/448] 10.222.0.47 () {56 vars in 1180 bytes} [Wed Aug 24 06:21:48 2022] GET /identity/v3/groups/9fb7ee19cac249789e1821689a52489e => generated 226 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:48.381373 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b98f586-169c-49db-b28a-af3d4701910d None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:48.415591 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea70a7f-af7f-4d9a-8d4d-f8782c5c7f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:48.416199 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea70a7f-af7f-4d9a-8d4d-f8782c5c7f1e None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:48.416479 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea70a7f-af7f-4d9a-8d4d-f8782c5c7f1e None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:48.416753 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea70a7f-af7f-4d9a-8d4d-f8782c5c7f1e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:48.437404 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:48.485557 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b98f586-169c-49db-b28a-af3d4701910d None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:48.487454 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea70a7f-af7f-4d9a-8d4d-f8782c5c7f1e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:48.488953 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea70a7f-af7f-4d9a-8d4d-f8782c5c7f1e None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:48.524364 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f453cd47-c7ed-416a-a9aa-f40ff94cf80d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:48.525373 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f453cd47-c7ed-416a-a9aa-f40ff94cf80d None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:48.525662 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f453cd47-c7ed-416a-a9aa-f40ff94cf80d None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:48.525929 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f453cd47-c7ed-416a-a9aa-f40ff94cf80d None admin] PATH_INFO: `/v3/groups/9fb7ee19cac249789e1821689a52489e` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:48.527039 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f453cd47-c7ed-416a-a9aa-f40ff94cf80d None admin] RBAC: Authorizing `identity:get_group(group_id=9fb7ee19cac249789e1821689a52489e)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:48.528762 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f453cd47-c7ed-416a-a9aa-f40ff94cf80d None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:48.535479 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 228/449] 10.222.0.47 () {56 vars in 1180 bytes} [Wed Aug 24 06:21:48 2022] GET /identity/v3/groups/9fb7ee19cac249789e1821689a52489e => generated 226 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:48.535479 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f453cd47-c7ed-416a-a9aa-f40ff94cf80d None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:48.551790 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:48.593876 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 222/450] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 24 06:21:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 06:21:48.594808 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea70a7f-af7f-4d9a-8d4d-f8782c5c7f1e None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:48.636635 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f453cd47-c7ed-416a-a9aa-f40ff94cf80d None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:48.659663 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:48.674282 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f00b44-b533-49cc-936d-8fca5d6f4a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:48.674884 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f00b44-b533-49cc-936d-8fca5d6f4a98 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:48.675194 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f00b44-b533-49cc-936d-8fca5d6f4a98 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:48.675468 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f00b44-b533-49cc-936d-8fca5d6f4a98 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:48.676191 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f00b44-b533-49cc-936d-8fca5d6f4a98 None admin] RBAC: Adding query filter params (group.id=9fb7ee19cac249789e1821689a52489e, role.id=e1f9b167c51c4a388cbc9594db8a465d, scope.project.id=c2a0c48ef166451b912daa9e82ddcd09) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:48.676514 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f00b44-b533-49cc-936d-8fca5d6f4a98 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:48.677972 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f00b44-b533-49cc-936d-8fca5d6f4a98 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:48.684705 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 229/451] 10.222.0.47 () {56 vars in 1376 bytes} [Wed Aug 24 06:21:48 2022] GET /identity/v3/role_assignments?group.id=9fb7ee19cac249789e1821689a52489e&scope.project.id=c2a0c48ef166451b912daa9e82ddcd09&role.id=e1f9b167c51c4a388cbc9594db8a465d => generated 611 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:48.686342 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f00b44-b533-49cc-936d-8fca5d6f4a98 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:48.703039 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea70a7f-af7f-4d9a-8d4d-f8782c5c7f1e None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:48.742827 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf84ba60-9baa-452e-8bbc-a2b46c21a532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:48.743305 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf84ba60-9baa-452e-8bbc-a2b46c21a532 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:48.743585 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf84ba60-9baa-452e-8bbc-a2b46c21a532 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:48.743851 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf84ba60-9baa-452e-8bbc-a2b46c21a532 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:48.744594 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf84ba60-9baa-452e-8bbc-a2b46c21a532 None admin] RBAC: Adding query filter params (group.id=9fb7ee19cac249789e1821689a52489e, role.id=b71c87e12dc1452ea3c99a7bf28585ca, scope.project.id=c2a0c48ef166451b912daa9e82ddcd09) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:48.744899 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf84ba60-9baa-452e-8bbc-a2b46c21a532 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:48.747007 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf84ba60-9baa-452e-8bbc-a2b46c21a532 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:48.765217 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 223/452] 10.222.0.47 () {56 vars in 1376 bytes} [Wed Aug 24 06:21:48 2022] GET /identity/v3/role_assignments?group.id=9fb7ee19cac249789e1821689a52489e&scope.project.id=c2a0c48ef166451b912daa9e82ddcd09&role.id=b71c87e12dc1452ea3c99a7bf28585ca => generated 611 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:48.824997 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f00b44-b533-49cc-936d-8fca5d6f4a98 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:48.867068 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0902a43a-e30a-4300-a113-9f654287b4e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:48.867725 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0902a43a-e30a-4300-a113-9f654287b4e4 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:48.868008 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0902a43a-e30a-4300-a113-9f654287b4e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:48.868292 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0902a43a-e30a-4300-a113-9f654287b4e4 None admin] PATH_INFO: `/v3/projects/24c1e9e70e07496595a602ca047e6a1f` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:48.869525 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0902a43a-e30a-4300-a113-9f654287b4e4 None admin] RBAC: Authorizing `identity:get_project(project_id=24c1e9e70e07496595a602ca047e6a1f)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:48.870994 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0902a43a-e30a-4300-a113-9f654287b4e4 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:48.873957 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 230/453] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 24 06:21:48 2022] GET /identity/v3/projects/24c1e9e70e07496595a602ca047e6a1f => generated 344 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:48.879560 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf84ba60-9baa-452e-8bbc-a2b46c21a532 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:48.912764 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:48.995082 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf84ba60-9baa-452e-8bbc-a2b46c21a532 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:49.031367 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d760074-91bc-4df8-89be-e494c1bc47bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:49.032004 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d760074-91bc-4df8-89be-e494c1bc47bc None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:49.032282 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d760074-91bc-4df8-89be-e494c1bc47bc None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:49.032553 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d760074-91bc-4df8-89be-e494c1bc47bc None admin] PATH_INFO: `/v3/groups/2faefd426a0748d6a8263d0eb0102204` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:49.033714 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d760074-91bc-4df8-89be-e494c1bc47bc None admin] RBAC: Authorizing `identity:get_group(group_id=2faefd426a0748d6a8263d0eb0102204)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:49.036866 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d760074-91bc-4df8-89be-e494c1bc47bc None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:49.036866 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 224/454] 10.222.0.47 () {56 vars in 1180 bytes} [Wed Aug 24 06:21:48 2022] GET /identity/v3/groups/2faefd426a0748d6a8263d0eb0102204 => generated 229 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:49.042551 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0902a43a-e30a-4300-a113-9f654287b4e4 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:49.101140 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:49.149379 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0902a43a-e30a-4300-a113-9f654287b4e4 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:49.184628 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c307678-b284-4861-be13-c3863c76c425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:49.185266 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c307678-b284-4861-be13-c3863c76c425 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:49.185544 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c307678-b284-4861-be13-c3863c76c425 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:49.185815 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c307678-b284-4861-be13-c3863c76c425 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:49.186540 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c307678-b284-4861-be13-c3863c76c425 None admin] RBAC: Adding query filter params (group.id=2faefd426a0748d6a8263d0eb0102204, role.id=486144ad1bcd417e86acdbc1d14327bf, scope.project.id=24c1e9e70e07496595a602ca047e6a1f) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:49.186844 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c307678-b284-4861-be13-c3863c76c425 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:49.188275 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c307678-b284-4861-be13-c3863c76c425 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:49.194342 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 231/455] 10.222.0.47 () {56 vars in 1376 bytes} [Wed Aug 24 06:21:49 2022] GET /identity/v3/role_assignments?group.id=2faefd426a0748d6a8263d0eb0102204&scope.project.id=24c1e9e70e07496595a602ca047e6a1f&role.id=486144ad1bcd417e86acdbc1d14327bf => generated 611 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:51.681264 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0228b183-e000-4582-bcc6-30f35f76173a None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:51.681805 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0228b183-e000-4582-bcc6-30f35f76173a None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:51.681805 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0228b183-e000-4582-bcc6-30f35f76173a None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:51.683492 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 225/456] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:21:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:21:51.689726 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295919e8-b1dd-454f-8e31-3d4fd91b1b9b None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:51.690637 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295919e8-b1dd-454f-8e31-3d4fd91b1b9b None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:51.691116 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295919e8-b1dd-454f-8e31-3d4fd91b1b9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:51.722239 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-295919e8-b1dd-454f-8e31-3d4fd91b1b9b None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:21:51.723898 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-295919e8-b1dd-454f-8e31-3d4fd91b1b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:21:51.803448 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeacf7f-4849-4729-ba6e-e37a2d064f14 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:51.804076 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeacf7f-4849-4729-ba6e-e37a2d064f14 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:51.804569 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeacf7f-4849-4729-ba6e-e37a2d064f14 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:51.806527 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 226/457] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:21:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:21:51.811651 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3848c3fc-d26a-441e-ab26-b4df053d5446 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:51.812216 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3848c3fc-d26a-441e-ab26-b4df053d5446 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:51.812712 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3848c3fc-d26a-441e-ab26-b4df053d5446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:51.824733 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 232/458] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:21:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:21:51.840900 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-3848c3fc-d26a-441e-ab26-b4df053d5446 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:21:51.842144 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-3848c3fc-d26a-441e-ab26-b4df053d5446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:21:51.865282 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f566483d-c8b8-4fed-af62-c243da2e0cfe None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:51.865282 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f566483d-c8b8-4fed-af62-c243da2e0cfe None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:51.865282 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f566483d-c8b8-4fed-af62-c243da2e0cfe None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:51.866323 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 233/459] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:21:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 06:21:51.873729 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32edf5d-f78d-4a99-b49a-b668ec38c70b None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:51.874288 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32edf5d-f78d-4a99-b49a-b668ec38c70b None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:51.874782 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32edf5d-f78d-4a99-b49a-b668ec38c70b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:51.897798 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-b32edf5d-f78d-4a99-b49a-b668ec38c70b None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:21:51.900302 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-b32edf5d-f78d-4a99-b49a-b668ec38c70b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:21:51.969569 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:51.994488 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 227/460] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:21:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 06:21:52.009333 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 234/461] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:21:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 06:21:52.164281 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3848c3fc-d26a-441e-ab26-b4df053d5446 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:52.200944 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:52.282836 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3848c3fc-d26a-441e-ab26-b4df053d5446 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:52.321457 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10fbb7a-e25e-4fab-9b31-e05729321816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:52.322111 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10fbb7a-e25e-4fab-9b31-e05729321816 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:52.322394 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10fbb7a-e25e-4fab-9b31-e05729321816 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:52.322697 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10fbb7a-e25e-4fab-9b31-e05729321816 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:52.335053 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10fbb7a-e25e-4fab-9b31-e05729321816 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:52.336519 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10fbb7a-e25e-4fab-9b31-e05729321816 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:52.353226 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-d10fbb7a-e25e-4fab-9b31-e05729321816 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:21:52.353226 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:21:52.353226 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:21:52.353226 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:21:52.353226 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:21:52.353226 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:21:52.353226 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:21:52.353226 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:21:52.353226 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:21:52.353226 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:21:52.353226 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:21:52.353226 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:21:52.353226 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:21:52.353226 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:21:52.353226 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:21:52.353226 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:21:52.354228 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:21:52.354228 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:21:52.354228 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:21:52.354228 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:21:52.354228 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:21:52.354228 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:21:52.354228 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:21:52.354228 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:21:52.354228 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:21:52.354228 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:21:52.354228 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:21:52.354228 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:21:52.354228 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:21:52.354228 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:21:52.354228 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:21:52.354228 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:21:52.354228 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:21:52.355479 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:21:52.355479 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:21:52.355479 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:21:52.355479 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 228/462] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:21:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:21:52.361978 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32edf5d-f78d-4a99-b49a-b668ec38c70b None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:52.384301 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10fbb7a-e25e-4fab-9b31-e05729321816 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:52.497756 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32edf5d-f78d-4a99-b49a-b668ec38c70b None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:52.501955 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:52.535475 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10fbb7a-e25e-4fab-9b31-e05729321816 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:52.537340 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc43ed2-0c1c-491e-8df2-5e0f26813a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:52.538057 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc43ed2-0c1c-491e-8df2-5e0f26813a7c None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:52.538340 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc43ed2-0c1c-491e-8df2-5e0f26813a7c None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:52.538612 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc43ed2-0c1c-491e-8df2-5e0f26813a7c None admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:52.539272 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc43ed2-0c1c-491e-8df2-5e0f26813a7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:52.539570 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc43ed2-0c1c-491e-8df2-5e0f26813a7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:52.540981 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc43ed2-0c1c-491e-8df2-5e0f26813a7c None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:52.554750 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 235/463] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 24 06:21:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:52.555627 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc43ed2-0c1c-491e-8df2-5e0f26813a7c None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:52.579959 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cb3e55-0480-408b-a1e8-18cce42e7089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:52.580611 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cb3e55-0480-408b-a1e8-18cce42e7089 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:52.580890 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cb3e55-0480-408b-a1e8-18cce42e7089 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:52.581165 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cb3e55-0480-408b-a1e8-18cce42e7089 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:52.595867 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cb3e55-0480-408b-a1e8-18cce42e7089 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:52.597362 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cb3e55-0480-408b-a1e8-18cce42e7089 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:52.604115 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:52.620411 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-79cb3e55-0480-408b-a1e8-18cce42e7089 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:21:52.620411 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:21:52.620411 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:21:52.620411 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:21:52.620411 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:21:52.620411 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:21:52.620411 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:21:52.620411 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:21:52.620411 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:21:52.620411 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:21:52.620411 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:21:52.620411 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:21:52.620411 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:21:52.620411 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:21:52.620411 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:21:52.620411 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:21:52.622651 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:21:52.622651 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:21:52.622651 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:21:52.622651 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:21:52.622651 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:21:52.622651 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:21:52.622651 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:21:52.622651 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:21:52.622651 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:21:52.622651 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:21:52.622651 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:21:52.622651 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:21:52.622651 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:21:52.622651 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:21:52.622651 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:21:52.622651 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:21:52.622651 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:21:52.633670 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:21:52.633670 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:21:52.633670 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:21:52.633670 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 229/464] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:21:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:21:52.633670 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cb3e55-0480-408b-a1e8-18cce42e7089 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:52.696569 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc43ed2-0c1c-491e-8df2-5e0f26813a7c None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:52.735941 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e18c8c-2f37-45a1-b447-e3d27e3a1906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:52.736590 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e18c8c-2f37-45a1-b447-e3d27e3a1906 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:52.736904 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e18c8c-2f37-45a1-b447-e3d27e3a1906 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:52.737207 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e18c8c-2f37-45a1-b447-e3d27e3a1906 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:52.750608 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e18c8c-2f37-45a1-b447-e3d27e3a1906 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:52.752275 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e18c8c-2f37-45a1-b447-e3d27e3a1906 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:52.756112 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cb3e55-0480-408b-a1e8-18cce42e7089 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:52.769147 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-f5e18c8c-2f37-45a1-b447-e3d27e3a1906 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:21:52.769147 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:21:52.769147 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:21:52.769147 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:21:52.769147 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:21:52.769147 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:21:52.769147 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:21:52.769147 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:21:52.769147 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:21:52.769147 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:21:52.769147 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:21:52.769147 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:21:52.769147 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:21:52.769147 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:21:52.769147 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:21:52.769147 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:21:52.777419 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:21:52.777419 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:21:52.777419 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:21:52.777419 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:21:52.777419 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:21:52.777419 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:21:52.777419 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:21:52.777419 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:21:52.777419 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:21:52.777419 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:21:52.777419 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:21:52.777419 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:21:52.777419 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:21:52.777419 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:21:52.777419 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:21:52.777419 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:21:52.777419 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:21:52.778292 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:21:52.778292 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:21:52.778292 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:21:52.778292 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 236/465] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:21:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:21:52.778292 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e18c8c-2f37-45a1-b447-e3d27e3a1906 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:52.789319 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:52.797537 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efd1715-d54f-4c62-b7df-60bc97fde55c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:52.798155 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efd1715-d54f-4c62-b7df-60bc97fde55c None admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:52.818485 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efd1715-d54f-4c62-b7df-60bc97fde55c None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:52.818485 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efd1715-d54f-4c62-b7df-60bc97fde55c None admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:52.818485 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efd1715-d54f-4c62-b7df-60bc97fde55c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:52.818485 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efd1715-d54f-4c62-b7df-60bc97fde55c None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:52.870026 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:52.882237 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e18c8c-2f37-45a1-b447-e3d27e3a1906 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:52.916265 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153a473e-06a4-42a3-9f70-0084df50b812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:52.916923 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153a473e-06a4-42a3-9f70-0084df50b812 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:52.917225 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153a473e-06a4-42a3-9f70-0084df50b812 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:52.917505 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153a473e-06a4-42a3-9f70-0084df50b812 None admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:52.918845 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153a473e-06a4-42a3-9f70-0084df50b812 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:52.919150 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153a473e-06a4-42a3-9f70-0084df50b812 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:52.920555 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153a473e-06a4-42a3-9f70-0084df50b812 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:52.931920 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 237/466] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 24 06:21:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:52.934275 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4da6475-c1db-4bd3-b794-ac9f10b26e78 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:52.934275 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4da6475-c1db-4bd3-b794-ac9f10b26e78 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:52.947300 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4da6475-c1db-4bd3-b794-ac9f10b26e78 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:52.947585 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 230/467] 10.222.0.47 () {60 vars in 1132 bytes} [Wed Aug 24 06:21:52 2022] POST /identity/v3/users => generated 246 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:21:52.947793 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 238/468] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:21:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:21:52.947888 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efd1715-d54f-4c62-b7df-60bc97fde55c None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:52.947971 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b155f9-ef4e-4df0-8d65-b196285452bf None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:52.947971 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b155f9-ef4e-4df0-8d65-b196285452bf None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:52.947971 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b155f9-ef4e-4df0-8d65-b196285452bf None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:52.947971 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 239/469] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:21:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:21:52.947971 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b155f9-ef4e-4df0-8d65-b196285452bf None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:53.060384 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efd1715-d54f-4c62-b7df-60bc97fde55c None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:53.060596 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b155f9-ef4e-4df0-8d65-b196285452bf None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:53.100759 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5992e8-3c1d-4db3-850e-55f7f72432f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:53.100973 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e680b984-3220-4678-a376-43850b5a8842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:53.101397 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5992e8-3c1d-4db3-850e-55f7f72432f0 None admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:53.101601 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e680b984-3220-4678-a376-43850b5a8842 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:53.101770 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5992e8-3c1d-4db3-850e-55f7f72432f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:53.101921 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e680b984-3220-4678-a376-43850b5a8842 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:53.102062 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5992e8-3c1d-4db3-850e-55f7f72432f0 None admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:53.102200 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e680b984-3220-4678-a376-43850b5a8842 None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:53.102566 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5992e8-3c1d-4db3-850e-55f7f72432f0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:53.102726 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e680b984-3220-4678-a376-43850b5a8842 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:53.102971 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e680b984-3220-4678-a376-43850b5a8842 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:53.103996 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5992e8-3c1d-4db3-850e-55f7f72432f0 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:53.104327 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e680b984-3220-4678-a376-43850b5a8842 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:53.117229 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 231/470] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 24 06:21:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:53.118460 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811ff2ad-6ed7-4921-b369-02968a78104b None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:53.118743 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811ff2ad-6ed7-4921-b369-02968a78104b None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:53.119015 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811ff2ad-6ed7-4921-b369-02968a78104b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:53.137675 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-811ff2ad-6ed7-4921-b369-02968a78104b None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:21:53.138928 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-811ff2ad-6ed7-4921-b369-02968a78104b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:21:53.179665 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:53.204382 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 240/471] 10.222.0.47 () {60 vars in 1132 bytes} [Wed Aug 24 06:21:52 2022] POST /identity/v3/users => generated 248 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:21:53.213211 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 232/472] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:21:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:21:53.213211 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811ff2ad-6ed7-4921-b369-02968a78104b None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:53.214085 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e257f636-251a-4011-a4c5-678e821d34b4 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:53.214532 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e257f636-251a-4011-a4c5-678e821d34b4 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:53.215529 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e257f636-251a-4011-a4c5-678e821d34b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:53.236420 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:53.256580 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-e257f636-251a-4011-a4c5-678e821d34b4 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:21:53.259257 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-e257f636-251a-4011-a4c5-678e821d34b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:21:53.397338 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811ff2ad-6ed7-4921-b369-02968a78104b None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:53.412949 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 241/473] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:21:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:21:53.443822 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73df865c-6629-4ca6-a96a-9b6bedb57892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:53.444448 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73df865c-6629-4ca6-a96a-9b6bedb57892 None admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:53.444725 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73df865c-6629-4ca6-a96a-9b6bedb57892 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:53.445001 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73df865c-6629-4ca6-a96a-9b6bedb57892 None admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:53.445962 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73df865c-6629-4ca6-a96a-9b6bedb57892 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:53.447166 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73df865c-6629-4ca6-a96a-9b6bedb57892 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:53.539575 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e257f636-251a-4011-a4c5-678e821d34b4 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:53.561746 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 233/474] 10.222.0.47 () {60 vars in 1132 bytes} [Wed Aug 24 06:21:53 2022] POST /identity/v3/users => generated 248 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:21:53.681931 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e257f636-251a-4011-a4c5-678e821d34b4 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:53.700904 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73df865c-6629-4ca6-a96a-9b6bedb57892 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:53.728117 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bacf3ca-ea76-4ee0-a8f0-372850cf42ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:53.728772 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bacf3ca-ea76-4ee0-a8f0-372850cf42ff None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:53.729052 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bacf3ca-ea76-4ee0-a8f0-372850cf42ff None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:53.729359 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bacf3ca-ea76-4ee0-a8f0-372850cf42ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:53.732127 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:53.742150 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bacf3ca-ea76-4ee0-a8f0-372850cf42ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:53.744342 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bacf3ca-ea76-4ee0-a8f0-372850cf42ff None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:53.762577 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:53.766953 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-8bacf3ca-ea76-4ee0-a8f0-372850cf42ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:21:53.766953 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:21:53.766953 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:21:53.766953 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:21:53.766953 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:21:53.766953 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:21:53.766953 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:21:53.766953 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:21:53.766953 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:21:53.766953 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:21:53.766953 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:21:53.766953 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:21:53.766953 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:21:53.766953 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:21:53.766953 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:21:53.766953 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:21:53.767872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:21:53.767872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:21:53.767872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:21:53.767872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:21:53.767872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:21:53.767872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:21:53.767872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:21:53.767872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:21:53.767872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:21:53.767872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:21:53.767872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:21:53.767872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:21:53.767872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:21:53.767872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:21:53.767872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:21:53.767872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:21:53.767872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:21:53.768715 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:21:53.768715 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:21:53.768715 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:21:53.769028 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 242/475] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:21:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:21:53.779086 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bacf3ca-ea76-4ee0-a8f0-372850cf42ff None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:53.843282 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73df865c-6629-4ca6-a96a-9b6bedb57892 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:53.885678 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d275e3-3538-4fbf-8d38-5793a60c31ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:53.887731 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d275e3-3538-4fbf-8d38-5793a60c31ee None admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:53.888413 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d275e3-3538-4fbf-8d38-5793a60c31ee None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:53.889071 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d275e3-3538-4fbf-8d38-5793a60c31ee None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:53.890343 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d275e3-3538-4fbf-8d38-5793a60c31ee None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:53.893091 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d275e3-3538-4fbf-8d38-5793a60c31ee None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:53.913892 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bacf3ca-ea76-4ee0-a8f0-372850cf42ff None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:53.934895 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.sql.core [None req-63d275e3-3538-4fbf-8d38-5793a60c31ee None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 06:21:53.934895 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 24 06:21:53.934895 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [parameters: {'id': 'f33f37be7c044151bfabdfd481121498', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 24 06:21:53.934895 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81724) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 06:21:53.955647 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe55422-d005-4a6d-8733-39b59ed5ed0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:53.956564 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe55422-d005-4a6d-8733-39b59ed5ed0b None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:53.956943 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe55422-d005-4a6d-8733-39b59ed5ed0b None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:53.957303 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe55422-d005-4a6d-8733-39b59ed5ed0b None admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:53.958029 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe55422-d005-4a6d-8733-39b59ed5ed0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:53.958454 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe55422-d005-4a6d-8733-39b59ed5ed0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:53.959946 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe55422-d005-4a6d-8733-39b59ed5ed0b None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:53.964335 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-63d275e3-3538-4fbf-8d38-5793a60c31ee None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 24 06:21:53.964335 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:21:53.964335 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 06:21:53.964335 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 06:21:53.964335 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 06:21:53.964335 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 06:21:53.964335 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 06:21:53.964335 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 06:21:53.964335 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 06:21:53.964335 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application conn.query(q) Aug 24 06:21:53.964335 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 06:21:53.964335 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 06:21:53.964335 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 06:21:53.964335 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application result.read() Aug 24 06:21:53.964335 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 06:21:53.965999 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 06:21:53.965999 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 06:21:53.965999 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 06:21:53.965999 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 06:21:53.965999 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 06:21:53.965999 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 06:21:53.965999 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 06:21:53.965999 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 06:21:53.965999 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:21:53.965999 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 06:21:53.965999 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:21:53.965999 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:21:53.965999 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 06:21:53.965999 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 06:21:53.965999 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 24 06:21:53.965999 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return ref.to_dict() Aug 24 06:21:53.965999 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 06:21:53.965999 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application next(self.gen) Aug 24 06:21:53.967278 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 06:21:53.967278 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application yield resource Aug 24 06:21:53.967278 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 06:21:53.967278 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application next(self.gen) Aug 24 06:21:53.967278 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 06:21:53.967278 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 06:21:53.967278 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 06:21:53.967278 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 06:21:53.967278 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 06:21:53.967278 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise self.value Aug 24 06:21:53.967278 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 06:21:53.967278 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 06:21:53.967278 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 06:21:53.967278 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application session.commit() Aug 24 06:21:53.967278 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 06:21:53.967278 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 06:21:53.967278 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 06:21:53.968912 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 06:21:53.968912 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 06:21:53.968912 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application self.session.flush() Aug 24 06:21:53.968912 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 06:21:53.968912 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 06:21:53.968912 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 06:21:53.968912 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 06:21:53.968912 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 06:21:53.968912 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application compat.raise_( Aug 24 06:21:53.968912 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 06:21:53.968912 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception Aug 24 06:21:53.968912 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 06:21:53.968912 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 06:21:53.968912 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 06:21:53.968912 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 06:21:53.968912 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 06:21:53.968912 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 06:21:53.972161 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 06:21:53.972161 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 06:21:53.972161 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 24 06:21:53.972161 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 24 06:21:53.972161 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 06:21:53.972161 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 06:21:53.972161 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 06:21:53.972161 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 06:21:53.972161 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 06:21:53.972161 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 06:21:53.972161 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 06:21:53.972161 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 06:21:53.972161 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 06:21:53.972161 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 06:21:53.972161 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 06:21:53.972161 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception Aug 24 06:21:53.974372 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 06:21:53.974372 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 06:21:53.974372 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 06:21:53.974372 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 06:21:53.974372 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 06:21:53.974372 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 06:21:53.974372 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 06:21:53.974372 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application conn.query(q) Aug 24 06:21:53.974372 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 06:21:53.974372 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 06:21:53.974372 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 06:21:53.974372 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application result.read() Aug 24 06:21:53.974372 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 06:21:53.974372 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 06:21:53.974372 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 06:21:53.974372 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 06:21:53.976694 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 06:21:53.976694 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 06:21:53.976694 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 06:21:53.976694 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 06:21:53.976694 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 06:21:53.976694 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 24 06:21:53.976694 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [parameters: {'id': 'f33f37be7c044151bfabdfd481121498', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 24 06:21:53.976694 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 06:21:53.976694 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:21:53.976694 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 06:21:53.976694 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:21:53.976694 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:21:53.976694 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:21:53.976694 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:21:53.976694 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:21:53.976694 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:21:53.981823 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:21:53.981823 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:21:53.981823 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:21:53.981823 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:21:53.981823 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:21:53.981823 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:21:53.981823 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 24 06:21:53.981823 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 24 06:21:53.981823 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:21:53.981823 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:21:53.981823 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 24 06:21:53.981823 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 24 06:21:53.981823 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 06:21:53.981823 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 06:21:53.981823 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 24 06:21:53.981823 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:21:53.982653 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 243/476] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 24 06:21:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:53.983421 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 234/477] 10.222.0.47 () {60 vars in 1131 bytes} [Wed Aug 24 06:21:53 2022] POST /identity/v3/roles => generated 144 bytes in 283 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:21:53.986159 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe55422-d005-4a6d-8733-39b59ed5ed0b None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:53.988840 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d275e3-3538-4fbf-8d38-5793a60c31ee None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:54.063182 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:54.137858 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe55422-d005-4a6d-8733-39b59ed5ed0b None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:54.150394 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:54.158612 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d275e3-3538-4fbf-8d38-5793a60c31ee None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:54.187063 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e324660-2065-4018-a669-ea03459cdd97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:54.192152 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e324660-2065-4018-a669-ea03459cdd97 None admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:54.193747 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e324660-2065-4018-a669-ea03459cdd97 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:54.194206 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e324660-2065-4018-a669-ea03459cdd97 None admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:54.195189 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e324660-2065-4018-a669-ea03459cdd97 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:54.198100 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e324660-2065-4018-a669-ea03459cdd97 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:54.222743 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e0c1d7-9694-40d6-9026-c82cbba89f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:54.223749 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e0c1d7-9694-40d6-9026-c82cbba89f7c None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:54.224451 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e0c1d7-9694-40d6-9026-c82cbba89f7c None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:54.225519 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e0c1d7-9694-40d6-9026-c82cbba89f7c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:54.240431 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e0c1d7-9694-40d6-9026-c82cbba89f7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:21:54.249020 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-a8e0c1d7-9694-40d6-9026-c82cbba89f7c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 06:21:54.249020 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:21:54.249020 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:21:54.249020 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:21:54.249020 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:21:54.249020 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:21:54.249020 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:21:54.249020 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:21:54.249020 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:21:54.249020 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:21:54.249020 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:21:54.249020 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:21:54.249020 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:21:54.249020 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:21:54.249020 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:21:54.249020 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:21:54.250186 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:21:54.250186 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:21:54.250186 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:21:54.250186 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:21:54.250186 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:21:54.250186 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:21:54.250186 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:21:54.250186 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:21:54.250186 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:21:54.250186 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:21:54.250186 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:21:54.250186 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:21:54.250186 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:21:54.250186 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:21:54.250186 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:21:54.250186 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:21:54.251206 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:21:54.251206 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:21:54.251206 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:21:54.251206 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:21:54.251206 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:21:54.251206 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:21:54.251206 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:21:54.251206 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:21:54.251206 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:21:54.251206 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:21:54.251206 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 06:21:54.251206 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:21:54.253605 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 235/478] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Aug 24 06:21:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:21:54.259497 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e0c1d7-9694-40d6-9026-c82cbba89f7c None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:54.308495 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 244/479] 10.222.0.47 () {60 vars in 1132 bytes} [Wed Aug 24 06:21:53 2022] POST /identity/v3/users => generated 249 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:21:54.388141 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e0c1d7-9694-40d6-9026-c82cbba89f7c None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:54.432991 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d852678-1e0c-43ea-92c6-e04b719f34a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:54.433857 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d852678-1e0c-43ea-92c6-e04b719f34a2 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:54.434268 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d852678-1e0c-43ea-92c6-e04b719f34a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:54.434695 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d852678-1e0c-43ea-92c6-e04b719f34a2 None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:54.435483 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d852678-1e0c-43ea-92c6-e04b719f34a2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:54.435887 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d852678-1e0c-43ea-92c6-e04b719f34a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:54.437529 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d852678-1e0c-43ea-92c6-e04b719f34a2 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:54.447214 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 236/480] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 24 06:21:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:21:55.387017 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db2f445-5d05-4dc2-9e16-134236909723 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:55.387667 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db2f445-5d05-4dc2-9e16-134236909723 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:55.387916 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db2f445-5d05-4dc2-9e16-134236909723 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:55.389733 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 245/481] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:21:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:21:55.395344 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db49901d-9ac0-4679-8933-dcf5deac032d None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:55.395796 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db49901d-9ac0-4679-8933-dcf5deac032d None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:55.395893 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db49901d-9ac0-4679-8933-dcf5deac032d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:55.422274 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-db49901d-9ac0-4679-8933-dcf5deac032d None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:21:55.423502 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-db49901d-9ac0-4679-8933-dcf5deac032d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:21:55.441521 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:55.509018 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 237/482] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:21:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:21:55.628253 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45aff85-c4a9-43ca-a1fe-4f5b9913eb56 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:55.628542 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45aff85-c4a9-43ca-a1fe-4f5b9913eb56 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:55.628823 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45aff85-c4a9-43ca-a1fe-4f5b9913eb56 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:55.630478 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 246/483] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:21:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:21:55.635684 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fa2cfc-c523-4475-aa12-f25c0261404a None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:55.636219 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fa2cfc-c523-4475-aa12-f25c0261404a None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:55.636787 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fa2cfc-c523-4475-aa12-f25c0261404a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:55.667134 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-a3fa2cfc-c523-4475-aa12-f25c0261404a None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:21:55.669595 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-a3fa2cfc-c523-4475-aa12-f25c0261404a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:21:55.750932 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 238/484] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:21:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:21:55.861460 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45aff85-c4a9-43ca-a1fe-4f5b9913eb56 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:56.010397 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45aff85-c4a9-43ca-a1fe-4f5b9913eb56 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:56.012767 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:56.055977 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a019f9-3cde-4cb6-9ad2-490f58fcc542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:56.056615 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a019f9-3cde-4cb6-9ad2-490f58fcc542 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:56.056899 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a019f9-3cde-4cb6-9ad2-490f58fcc542 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:56.057191 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a019f9-3cde-4cb6-9ad2-490f58fcc542 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:56.078148 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a019f9-3cde-4cb6-9ad2-490f58fcc542 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:21:56.083322 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-d9a019f9-3cde-4cb6-9ad2-490f58fcc542 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:21:56.083322 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:21:56.083322 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:21:56.083322 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:21:56.083322 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:21:56.083322 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:21:56.083322 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:21:56.083322 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:21:56.083322 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:21:56.083322 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:21:56.083322 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:21:56.083322 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:21:56.083322 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:21:56.083322 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:21:56.083322 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:21:56.083322 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:21:56.084290 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:21:56.084290 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:21:56.084290 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:21:56.084290 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:21:56.084290 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:21:56.084290 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:21:56.084290 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:21:56.084290 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:21:56.084290 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:21:56.084290 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:21:56.084290 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:21:56.084290 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:21:56.084290 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:21:56.084290 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:21:56.084290 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:21:56.084290 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:21:56.085092 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:21:56.085092 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:21:56.085092 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:21:56.085092 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:21:56.085092 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:21:56.085092 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:21:56.085092 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:21:56.085092 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:21:56.085092 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:21:56.085092 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:21:56.085092 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:21:56.085092 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:21:56.090882 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 247/485] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Aug 24 06:21:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:21:56.096141 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a019f9-3cde-4cb6-9ad2-490f58fcc542 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:56.096338 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fa2cfc-c523-4475-aa12-f25c0261404a None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:56.243230 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a019f9-3cde-4cb6-9ad2-490f58fcc542 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:56.243902 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:56.271509 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fa2cfc-c523-4475-aa12-f25c0261404a None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:56.284793 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf6a967-2392-42a4-a34a-abfb09696b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:56.285439 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf6a967-2392-42a4-a34a-abfb09696b52 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:56.285733 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf6a967-2392-42a4-a34a-abfb09696b52 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:56.286012 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf6a967-2392-42a4-a34a-abfb09696b52 None admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:56.286687 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf6a967-2392-42a4-a34a-abfb09696b52 None admin] RBAC: Adding query filter params (name=service) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:56.287140 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf6a967-2392-42a4-a34a-abfb09696b52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:56.288667 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf6a967-2392-42a4-a34a-abfb09696b52 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:56.298750 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 248/486] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Aug 24 06:21:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:21:56.299956 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e039ba-4cc9-45f1-8d4d-c1b60138453c None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:56.301295 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e039ba-4cc9-45f1-8d4d-c1b60138453c None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:56.302217 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e039ba-4cc9-45f1-8d4d-c1b60138453c None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:56.303474 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 249/487] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:21:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:21:56.305601 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e039ba-4cc9-45f1-8d4d-c1b60138453c None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:56.312657 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067ad772-6ced-4db8-a3dd-f523b7b6bfa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:56.313327 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067ad772-6ced-4db8-a3dd-f523b7b6bfa7 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:56.313611 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067ad772-6ced-4db8-a3dd-f523b7b6bfa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:56.313886 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067ad772-6ced-4db8-a3dd-f523b7b6bfa7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:56.324682 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067ad772-6ced-4db8-a3dd-f523b7b6bfa7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:21:56.329613 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-067ad772-6ced-4db8-a3dd-f523b7b6bfa7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:21:56.329613 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:21:56.329613 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:21:56.329613 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:21:56.329613 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:21:56.329613 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:21:56.329613 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:21:56.329613 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:21:56.329613 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:21:56.329613 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:21:56.329613 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:21:56.329613 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:21:56.329613 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:21:56.329613 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:21:56.329613 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:21:56.329613 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:21:56.330562 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:21:56.330562 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:21:56.330562 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:21:56.330562 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:21:56.330562 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:21:56.330562 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:21:56.330562 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:21:56.330562 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:21:56.330562 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:21:56.330562 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:21:56.330562 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:21:56.330562 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:21:56.330562 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:21:56.330562 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:21:56.330562 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:21:56.330562 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:21:56.331360 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:21:56.331360 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:21:56.331360 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:21:56.331360 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:21:56.331360 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:21:56.331360 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:21:56.331360 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:21:56.331360 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:21:56.331360 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:21:56.331360 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:21:56.331360 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:21:56.331360 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:21:56.335618 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 239/488] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Aug 24 06:21:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:21:56.336837 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416d44be-1994-43a9-a1be-d523e87812fe None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:56.337124 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416d44be-1994-43a9-a1be-d523e87812fe None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:56.337767 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416d44be-1994-43a9-a1be-d523e87812fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:56.341918 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:56.356603 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-416d44be-1994-43a9-a1be-d523e87812fe None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:21:56.357849 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-416d44be-1994-43a9-a1be-d523e87812fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:21:56.437800 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e039ba-4cc9-45f1-8d4d-c1b60138453c None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:56.446939 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 240/489] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:21:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:21:56.447794 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416d44be-1994-43a9-a1be-d523e87812fe None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:56.485286 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375de0e2-1c77-42ad-a128-7d7b07bcc278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:56.486321 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375de0e2-1c77-42ad-a128-7d7b07bcc278 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:56.486827 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375de0e2-1c77-42ad-a128-7d7b07bcc278 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:56.487340 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375de0e2-1c77-42ad-a128-7d7b07bcc278 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:56.514642 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375de0e2-1c77-42ad-a128-7d7b07bcc278 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:56.516147 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:56.516641 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375de0e2-1c77-42ad-a128-7d7b07bcc278 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:56.536193 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-375de0e2-1c77-42ad-a128-7d7b07bcc278 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:21:56.536193 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:21:56.536193 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:21:56.536193 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:21:56.536193 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:21:56.536193 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:21:56.536193 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:21:56.536193 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:21:56.536193 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:21:56.536193 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:21:56.536193 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:21:56.536193 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:21:56.536193 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:21:56.536193 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:21:56.536193 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:21:56.536193 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:21:56.537112 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:21:56.537112 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:21:56.537112 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:21:56.537112 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:21:56.537112 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:21:56.537112 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:21:56.537112 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:21:56.537112 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:21:56.537112 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:21:56.537112 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:21:56.537112 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:21:56.537112 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:21:56.537112 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:21:56.537112 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:21:56.537112 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:21:56.537112 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:21:56.537112 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:21:56.537976 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:21:56.537976 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:21:56.537976 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:21:56.546087 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 250/490] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:21:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:21:56.551963 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375de0e2-1c77-42ad-a128-7d7b07bcc278 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:56.559626 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416d44be-1994-43a9-a1be-d523e87812fe None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:56.595928 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d73d835-7747-49ae-9181-babf86797423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:56.596891 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d73d835-7747-49ae-9181-babf86797423 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:56.597514 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d73d835-7747-49ae-9181-babf86797423 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:56.597999 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d73d835-7747-49ae-9181-babf86797423 None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:56.598855 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d73d835-7747-49ae-9181-babf86797423 None admin] RBAC: Adding query filter params (name=service) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:56.599349 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d73d835-7747-49ae-9181-babf86797423 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:56.600972 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d73d835-7747-49ae-9181-babf86797423 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:56.609877 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 241/491] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Aug 24 06:21:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:56.615816 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d73d835-7747-49ae-9181-babf86797423 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:56.708394 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:56.750563 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375de0e2-1c77-42ad-a128-7d7b07bcc278 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:56.779949 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d73d835-7747-49ae-9181-babf86797423 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:56.796894 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d948ce39-5585-4d68-a708-89debfff9a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:56.797455 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d948ce39-5585-4d68-a708-89debfff9a59 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:56.797976 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d948ce39-5585-4d68-a708-89debfff9a59 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:56.798248 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d948ce39-5585-4d68-a708-89debfff9a59 None admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:56.799287 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d948ce39-5585-4d68-a708-89debfff9a59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:56.799588 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d948ce39-5585-4d68-a708-89debfff9a59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:56.800964 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d948ce39-5585-4d68-a708-89debfff9a59 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:56.809420 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:56.817737 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f4f2a9-1325-4e0a-b375-bfc98df12c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:56.818468 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 251/492] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 24 06:21:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:21:56.818591 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f4f2a9-1325-4e0a-b375-bfc98df12c52 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:56.818818 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f4f2a9-1325-4e0a-b375-bfc98df12c52 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:56.819173 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f4f2a9-1325-4e0a-b375-bfc98df12c52 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:56.819408 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d948ce39-5585-4d68-a708-89debfff9a59 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:56.833195 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f4f2a9-1325-4e0a-b375-bfc98df12c52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:56.834686 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f4f2a9-1325-4e0a-b375-bfc98df12c52 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:56.854519 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-e1f4f2a9-1325-4e0a-b375-bfc98df12c52 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:21:56.854519 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:21:56.854519 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:21:56.854519 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:21:56.854519 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:21:56.854519 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:21:56.854519 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:21:56.854519 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:21:56.854519 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:21:56.854519 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:21:56.854519 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:21:56.854519 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:21:56.854519 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:21:56.854519 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:21:56.854519 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:21:56.854519 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:21:56.855820 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:21:56.855820 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:21:56.855820 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:21:56.855820 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:21:56.855820 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:21:56.855820 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:21:56.855820 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:21:56.855820 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:21:56.855820 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:21:56.855820 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:21:56.855820 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:21:56.855820 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:21:56.855820 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:21:56.855820 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:21:56.855820 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:21:56.855820 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:21:56.855820 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:21:56.857047 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:21:56.857047 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:21:56.857047 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:21:56.857047 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 242/493] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:21:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:21:56.858364 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f4f2a9-1325-4e0a-b375-bfc98df12c52 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:56.943003 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d948ce39-5585-4d68-a708-89debfff9a59 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:56.977782 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f4f2a9-1325-4e0a-b375-bfc98df12c52 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:56.981380 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47a8993-92ea-438a-ab8b-ac00c1296a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:56.982160 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47a8993-92ea-438a-ab8b-ac00c1296a4f None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:56.982544 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47a8993-92ea-438a-ab8b-ac00c1296a4f None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:56.982918 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47a8993-92ea-438a-ab8b-ac00c1296a4f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:56.990438 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:56.998507 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47a8993-92ea-438a-ab8b-ac00c1296a4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:21:57.005103 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-a47a8993-92ea-438a-ab8b-ac00c1296a4f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:21:57.005103 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:21:57.005103 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:21:57.005103 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:21:57.005103 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:21:57.005103 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:21:57.005103 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:21:57.005103 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:21:57.005103 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:21:57.005103 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:21:57.005103 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:21:57.005103 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:21:57.005103 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:21:57.005103 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:21:57.005103 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:21:57.005103 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:21:57.006153 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:21:57.006153 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:21:57.006153 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:21:57.006153 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:21:57.006153 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:21:57.006153 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:21:57.006153 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:21:57.006153 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:21:57.006153 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:21:57.006153 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:21:57.006153 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:21:57.006153 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:21:57.006153 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:21:57.006153 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:21:57.006153 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:21:57.006153 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:21:57.006980 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:21:57.006980 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:21:57.006980 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:21:57.006980 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:21:57.006980 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:21:57.006980 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:21:57.006980 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:21:57.006980 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:21:57.006980 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:21:57.006980 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:21:57.006980 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:21:57.006980 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:21:57.012787 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 252/494] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Aug 24 06:21:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:21:57.014145 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47a8993-92ea-438a-ab8b-ac00c1296a4f None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:57.023127 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b635ecec-6800-4c39-b2b0-07eaf37e290a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:57.024125 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b635ecec-6800-4c39-b2b0-07eaf37e290a None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:57.024875 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b635ecec-6800-4c39-b2b0-07eaf37e290a None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:57.025457 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b635ecec-6800-4c39-b2b0-07eaf37e290a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:57.034371 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b635ecec-6800-4c39-b2b0-07eaf37e290a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:57.036164 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b635ecec-6800-4c39-b2b0-07eaf37e290a None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:57.053495 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-b635ecec-6800-4c39-b2b0-07eaf37e290a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 06:21:57.053495 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:21:57.053495 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:21:57.053495 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:21:57.053495 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:21:57.053495 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:21:57.053495 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:21:57.053495 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:21:57.053495 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:21:57.053495 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:21:57.053495 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:21:57.053495 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:21:57.053495 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 06:21:57.053495 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 06:21:57.053495 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 06:21:57.053495 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 06:21:57.054450 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:21:57.054450 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:21:57.054450 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 06:21:57.054450 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:21:57.054450 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 06:21:57.054450 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:21:57.054450 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:21:57.054450 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:21:57.054450 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:21:57.054450 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:21:57.054450 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:21:57.054450 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:21:57.054450 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:21:57.054450 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:21:57.054450 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:21:57.054450 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:21:57.054450 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:21:57.054450 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:21:57.055358 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:21:57.055358 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:21:57.055358 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 06:21:57.055358 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 06:21:57.055358 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 06:21:57.055358 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 06:21:57.055358 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 06:21:57.055358 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 06:21:57.055358 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 06:21:57.055358 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:21:57.055358 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 243/495] 10.222.0.47 () {56 vars in 1093 bytes} [Wed Aug 24 06:21:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:21:57.056041 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b635ecec-6800-4c39-b2b0-07eaf37e290a None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:57.128204 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47a8993-92ea-438a-ab8b-ac00c1296a4f None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:57.130163 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:57.166958 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11836525-147f-4560-9ece-78da8c31cd74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:57.167607 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11836525-147f-4560-9ece-78da8c31cd74 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:57.167892 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11836525-147f-4560-9ece-78da8c31cd74 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:57.168146 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11836525-147f-4560-9ece-78da8c31cd74 None admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:57.168738 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b635ecec-6800-4c39-b2b0-07eaf37e290a None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:57.168839 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11836525-147f-4560-9ece-78da8c31cd74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:57.169102 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11836525-147f-4560-9ece-78da8c31cd74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:57.170524 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11836525-147f-4560-9ece-78da8c31cd74 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:57.191204 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 253/496] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 24 06:21:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:57.191204 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ab495f-61a3-4ac5-a6cc-8d389c35b123 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:57.191204 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ab495f-61a3-4ac5-a6cc-8d389c35b123 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:57.191204 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ab495f-61a3-4ac5-a6cc-8d389c35b123 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:57.191204 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 254/497] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:21:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:21:57.191964 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ab495f-61a3-4ac5-a6cc-8d389c35b123 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:57.221786 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60080c7a-6ff2-4c95-a03e-10494c27515f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:57.222405 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60080c7a-6ff2-4c95-a03e-10494c27515f None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:57.222687 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60080c7a-6ff2-4c95-a03e-10494c27515f None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:57.222957 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60080c7a-6ff2-4c95-a03e-10494c27515f None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:57.223632 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60080c7a-6ff2-4c95-a03e-10494c27515f None admin] RBAC: Adding query filter params (name=service) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:57.223934 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60080c7a-6ff2-4c95-a03e-10494c27515f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:57.225474 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60080c7a-6ff2-4c95-a03e-10494c27515f None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:57.237235 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 244/498] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Aug 24 06:21:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:57.237235 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60080c7a-6ff2-4c95-a03e-10494c27515f None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:57.265681 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:57.313934 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ab495f-61a3-4ac5-a6cc-8d389c35b123 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:57.344155 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60080c7a-6ff2-4c95-a03e-10494c27515f None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:57.345781 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:57.350486 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3acf9b-e7fb-4887-ad2a-899a5053c1de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:57.351096 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3acf9b-e7fb-4887-ad2a-899a5053c1de None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:57.351375 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3acf9b-e7fb-4887-ad2a-899a5053c1de None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:57.351643 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3acf9b-e7fb-4887-ad2a-899a5053c1de None admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:57.352382 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3acf9b-e7fb-4887-ad2a-899a5053c1de None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:57.353106 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3acf9b-e7fb-4887-ad2a-899a5053c1de None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:57.354176 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3acf9b-e7fb-4887-ad2a-899a5053c1de None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:57.383342 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b784f59-3acd-44aa-ade9-78d3b9b9d134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:57.383980 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b784f59-3acd-44aa-ade9-78d3b9b9d134 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:57.384263 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b784f59-3acd-44aa-ade9-78d3b9b9d134 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:57.384646 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b784f59-3acd-44aa-ade9-78d3b9b9d134 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:57.393375 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b784f59-3acd-44aa-ade9-78d3b9b9d134 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:57.394865 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b784f59-3acd-44aa-ade9-78d3b9b9d134 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:57.395452 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 255/499] 10.222.0.47 () {56 vars in 1133 bytes} [Wed Aug 24 06:21:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:57.396484 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b4d5e2-576a-45e1-b23d-eee57fa4c1a8 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:57.396771 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b4d5e2-576a-45e1-b23d-eee57fa4c1a8 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:57.397046 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b4d5e2-576a-45e1-b23d-eee57fa4c1a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:57.412634 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-4b784f59-3acd-44aa-ade9-78d3b9b9d134 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 06:21:57.412634 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:21:57.412634 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:21:57.412634 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:21:57.412634 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:21:57.412634 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:21:57.412634 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:21:57.412634 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:21:57.412634 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:21:57.412634 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:21:57.412634 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:21:57.412634 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:21:57.412634 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 06:21:57.412634 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 06:21:57.412634 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 06:21:57.412634 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 06:21:57.413730 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:21:57.413730 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:21:57.413730 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 06:21:57.413730 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:21:57.413730 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 06:21:57.413730 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:21:57.413730 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:21:57.413730 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:21:57.413730 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:21:57.413730 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:21:57.413730 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:21:57.413730 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:21:57.413730 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:21:57.413730 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:21:57.413730 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:21:57.413730 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:21:57.413730 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:21:57.413730 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:21:57.414611 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:21:57.414611 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:21:57.414611 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 06:21:57.414611 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 06:21:57.414611 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 06:21:57.414611 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 06:21:57.414611 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 06:21:57.414611 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 06:21:57.414611 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 06:21:57.414611 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:21:57.414611 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 245/500] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 24 06:21:57 2022] GET /identity/v3/users/cinder => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:21:57.415465 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72846c1d-ba63-4a66-bc07-6a1256a92f29 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:57.415465 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72846c1d-ba63-4a66-bc07-6a1256a92f29 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:57.415590 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-f9b4d5e2-576a-45e1-b23d-eee57fa4c1a8 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:21:57.415659 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72846c1d-ba63-4a66-bc07-6a1256a92f29 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:57.416723 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-f9b4d5e2-576a-45e1-b23d-eee57fa4c1a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:21:57.421406 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 246/501] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:21:57 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:21:57.422602 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72846c1d-ba63-4a66-bc07-6a1256a92f29 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:57.487437 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 256/502] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:21:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:21:57.488300 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b4d5e2-576a-45e1-b23d-eee57fa4c1a8 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:57.537886 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72846c1d-ba63-4a66-bc07-6a1256a92f29 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:57.548905 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:57.580166 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bf8e5b-b456-4ac1-afe8-67d0a94f1020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:57.580839 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bf8e5b-b456-4ac1-afe8-67d0a94f1020 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:57.581117 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bf8e5b-b456-4ac1-afe8-67d0a94f1020 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:57.581412 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bf8e5b-b456-4ac1-afe8-67d0a94f1020 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:57.592749 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bf8e5b-b456-4ac1-afe8-67d0a94f1020 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:57.594142 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bf8e5b-b456-4ac1-afe8-67d0a94f1020 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:57.610625 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-f0bf8e5b-b456-4ac1-afe8-67d0a94f1020 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:21:57.610625 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:21:57.610625 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:21:57.610625 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:21:57.610625 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:21:57.610625 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:21:57.610625 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:21:57.610625 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:21:57.610625 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:21:57.610625 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:21:57.610625 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:21:57.610625 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:21:57.610625 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:21:57.610625 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:21:57.610625 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:21:57.610625 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:21:57.611583 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:21:57.611583 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:21:57.611583 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:21:57.611583 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:21:57.611583 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:21:57.611583 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:21:57.611583 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:21:57.611583 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:21:57.611583 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:21:57.611583 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:21:57.611583 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:21:57.611583 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:21:57.611583 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:21:57.611583 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:21:57.611583 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:21:57.611583 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:21:57.611583 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:21:57.617464 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:21:57.617464 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:21:57.617464 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:21:57.617768 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b4d5e2-576a-45e1-b23d-eee57fa4c1a8 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:57.617907 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 247/503] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:21:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:21:57.617907 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bf8e5b-b456-4ac1-afe8-67d0a94f1020 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:57.636595 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:57.666995 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18b7e5d-5d94-4eac-87e7-0b2d77533e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:57.668244 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18b7e5d-5d94-4eac-87e7-0b2d77533e18 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:57.668610 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18b7e5d-5d94-4eac-87e7-0b2d77533e18 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:57.668956 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18b7e5d-5d94-4eac-87e7-0b2d77533e18 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:57.742358 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bf8e5b-b456-4ac1-afe8-67d0a94f1020 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:57.750918 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18b7e5d-5d94-4eac-87e7-0b2d77533e18 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:57.752970 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18b7e5d-5d94-4eac-87e7-0b2d77533e18 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:57.779651 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3b3844-6c76-400e-b544-188e2f8dd26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:57.780266 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3b3844-6c76-400e-b544-188e2f8dd26b None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:57.780541 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3b3844-6c76-400e-b544-188e2f8dd26b None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:57.780813 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3b3844-6c76-400e-b544-188e2f8dd26b None admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:57.781591 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3b3844-6c76-400e-b544-188e2f8dd26b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:57.781890 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3b3844-6c76-400e-b544-188e2f8dd26b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:57.783314 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3b3844-6c76-400e-b544-188e2f8dd26b None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:57.801204 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:57.823426 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 248/504] 10.222.0.47 () {56 vars in 1137 bytes} [Wed Aug 24 06:21:57 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:57.825020 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb3d865-308d-46cf-9ab6-4f7a54bd749a None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:57.825333 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb3d865-308d-46cf-9ab6-4f7a54bd749a None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:57.825604 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb3d865-308d-46cf-9ab6-4f7a54bd749a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:57.847798 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-ddb3d865-308d-46cf-9ab6-4f7a54bd749a None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:21:57.847798 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-ddb3d865-308d-46cf-9ab6-4f7a54bd749a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:21:57.862452 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 257/505] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 24 06:21:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 06:21:57.867813 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18b7e5d-5d94-4eac-87e7-0b2d77533e18 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:57.918551 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 249/506] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:21:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:21:57.919405 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb3d865-308d-46cf-9ab6-4f7a54bd749a None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:57.921637 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:57.973032 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18b7e5d-5d94-4eac-87e7-0b2d77533e18 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:58.011720 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293cb796-c639-443e-8a27-85021e062ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:58.012707 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293cb796-c639-443e-8a27-85021e062ffa None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:58.012978 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293cb796-c639-443e-8a27-85021e062ffa None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:58.013277 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293cb796-c639-443e-8a27-85021e062ffa None admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:58.013935 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293cb796-c639-443e-8a27-85021e062ffa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:58.014239 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293cb796-c639-443e-8a27-85021e062ffa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:58.015623 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293cb796-c639-443e-8a27-85021e062ffa None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:58.028231 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 258/507] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 24 06:21:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:58.029122 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293cb796-c639-443e-8a27-85021e062ffa None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:58.035049 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:58.049201 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb3d865-308d-46cf-9ab6-4f7a54bd749a None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:58.088781 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfc3fcf-8a47-4030-9ae9-7cee648f6446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:58.090311 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfc3fcf-8a47-4030-9ae9-7cee648f6446 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:58.090594 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfc3fcf-8a47-4030-9ae9-7cee648f6446 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:58.090892 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfc3fcf-8a47-4030-9ae9-7cee648f6446 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:58.102292 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfc3fcf-8a47-4030-9ae9-7cee648f6446 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:58.103870 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfc3fcf-8a47-4030-9ae9-7cee648f6446 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:58.119845 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-3cfc3fcf-8a47-4030-9ae9-7cee648f6446 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:21:58.119845 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:21:58.119845 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:21:58.119845 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:21:58.119845 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:21:58.119845 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:21:58.119845 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:21:58.119845 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:21:58.119845 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:21:58.119845 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:21:58.119845 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:21:58.119845 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:21:58.119845 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:21:58.119845 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:21:58.119845 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:21:58.119845 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:21:58.120884 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:21:58.120884 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:21:58.120884 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:21:58.120884 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:21:58.120884 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:21:58.120884 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:21:58.120884 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:21:58.120884 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:21:58.120884 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:21:58.120884 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:21:58.120884 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:21:58.120884 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:21:58.120884 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:21:58.120884 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:21:58.120884 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:21:58.120884 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:21:58.120884 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:21:58.121874 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:21:58.121874 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:21:58.121874 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:21:58.121874 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 250/508] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:21:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:21:58.122086 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfc3fcf-8a47-4030-9ae9-7cee648f6446 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:58.151312 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293cb796-c639-443e-8a27-85021e062ffa None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:58.181889 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:58.189153 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9fa241-7de8-4504-8006-0dae5727c791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:58.189791 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9fa241-7de8-4504-8006-0dae5727c791 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:58.190070 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9fa241-7de8-4504-8006-0dae5727c791 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:58.198885 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9fa241-7de8-4504-8006-0dae5727c791 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:58.226931 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfc3fcf-8a47-4030-9ae9-7cee648f6446 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:58.260407 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb89acbe-876b-423e-8fc4-afd19fcfebed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:58.261561 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb89acbe-876b-423e-8fc4-afd19fcfebed None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:58.261938 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb89acbe-876b-423e-8fc4-afd19fcfebed None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:58.262244 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb89acbe-876b-423e-8fc4-afd19fcfebed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:58.262365 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9fa241-7de8-4504-8006-0dae5727c791 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:58.263769 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9fa241-7de8-4504-8006-0dae5727c791 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:58.266697 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:58.273463 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb89acbe-876b-423e-8fc4-afd19fcfebed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:58.275162 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb89acbe-876b-423e-8fc4-afd19fcfebed None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:58.290781 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-fb89acbe-876b-423e-8fc4-afd19fcfebed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:21:58.290781 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:21:58.290781 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:21:58.290781 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:21:58.290781 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:21:58.290781 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:21:58.290781 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:21:58.290781 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:21:58.290781 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:21:58.290781 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:21:58.290781 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:21:58.290781 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:21:58.290781 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:21:58.290781 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:21:58.290781 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:21:58.290781 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:21:58.291964 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:21:58.291964 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:21:58.291964 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:21:58.291964 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:21:58.291964 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:21:58.291964 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:21:58.291964 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:21:58.291964 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:21:58.291964 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:21:58.291964 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:21:58.291964 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:21:58.291964 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:21:58.291964 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:21:58.291964 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:21:58.291964 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:21:58.291964 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:21:58.291964 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:21:58.292856 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:21:58.292856 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:21:58.292856 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:21:58.292856 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 251/509] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:21:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:21:58.293388 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb89acbe-876b-423e-8fc4-afd19fcfebed None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:58.376555 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 259/510] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 24 06:21:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 06:21:58.377454 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9fa241-7de8-4504-8006-0dae5727c791 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:58.404103 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb89acbe-876b-423e-8fc4-afd19fcfebed None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:58.411289 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:58.447934 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b13bc0-699f-4fc6-b1aa-52f213b039e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:58.449146 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b13bc0-699f-4fc6-b1aa-52f213b039e2 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:58.449482 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b13bc0-699f-4fc6-b1aa-52f213b039e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:58.449827 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b13bc0-699f-4fc6-b1aa-52f213b039e2 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:58.457717 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b13bc0-699f-4fc6-b1aa-52f213b039e2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:58.459165 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b13bc0-699f-4fc6-b1aa-52f213b039e2 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:58.471053 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-24b13bc0-699f-4fc6-b1aa-52f213b039e2 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 06:21:58.471053 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:21:58.471053 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:21:58.471053 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:21:58.471053 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:21:58.471053 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:21:58.471053 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:21:58.471053 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:21:58.471053 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:21:58.471053 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:21:58.471053 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:21:58.471053 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:21:58.471053 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 06:21:58.471053 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 06:21:58.471053 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 06:21:58.471053 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 06:21:58.472160 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:21:58.472160 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:21:58.472160 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 06:21:58.472160 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:21:58.472160 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 06:21:58.472160 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:21:58.472160 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:21:58.472160 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:21:58.472160 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:21:58.472160 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:21:58.472160 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:21:58.472160 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:21:58.472160 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:21:58.472160 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:21:58.472160 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:21:58.472160 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:21:58.472160 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:21:58.472160 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:21:58.473308 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:21:58.473308 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:21:58.473308 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 06:21:58.473308 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 06:21:58.473308 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 06:21:58.473308 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 06:21:58.473308 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 06:21:58.473308 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 06:21:58.473308 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 06:21:58.473308 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:21:58.473308 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 252/511] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 24 06:21:58 2022] GET /identity/v3/users/glance => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:21:58.473308 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b13bc0-699f-4fc6-b1aa-52f213b039e2 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:58.485205 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9fa241-7de8-4504-8006-0dae5727c791 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:58.488156 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:58.521206 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25416a65-3f0c-4c8b-8f9e-adbc4f3cf8d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:58.521842 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25416a65-3f0c-4c8b-8f9e-adbc4f3cf8d4 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:58.522138 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25416a65-3f0c-4c8b-8f9e-adbc4f3cf8d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:58.522425 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25416a65-3f0c-4c8b-8f9e-adbc4f3cf8d4 None admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:58.523069 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25416a65-3f0c-4c8b-8f9e-adbc4f3cf8d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:58.523375 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25416a65-3f0c-4c8b-8f9e-adbc4f3cf8d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:58.525041 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25416a65-3f0c-4c8b-8f9e-adbc4f3cf8d4 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:58.537332 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 260/512] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 24 06:21:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:58.538184 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25416a65-3f0c-4c8b-8f9e-adbc4f3cf8d4 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:58.574398 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b13bc0-699f-4fc6-b1aa-52f213b039e2 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:58.609199 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e26319-5fae-44d1-bd06-acac38b6ae94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:58.609958 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e26319-5fae-44d1-bd06-acac38b6ae94 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:58.610319 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e26319-5fae-44d1-bd06-acac38b6ae94 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:58.610729 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e26319-5fae-44d1-bd06-acac38b6ae94 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:58.625728 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e26319-5fae-44d1-bd06-acac38b6ae94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:21:58.631154 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-d7e26319-5fae-44d1-bd06-acac38b6ae94 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:21:58.631154 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:21:58.631154 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:21:58.631154 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:21:58.631154 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:21:58.631154 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:21:58.631154 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:21:58.631154 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:21:58.631154 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:21:58.631154 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:21:58.631154 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:21:58.631154 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:21:58.631154 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:21:58.631154 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:21:58.631154 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:21:58.631154 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:21:58.632120 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:21:58.632120 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:21:58.632120 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:21:58.632120 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:21:58.632120 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:21:58.632120 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:21:58.632120 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:21:58.632120 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:21:58.632120 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:21:58.632120 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:21:58.632120 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:21:58.632120 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:21:58.632120 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:21:58.632120 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:21:58.632120 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:21:58.632120 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:21:58.632929 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:21:58.632929 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:21:58.632929 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:21:58.632929 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:21:58.632929 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:21:58.632929 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:21:58.632929 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:21:58.632929 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:21:58.632929 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:21:58.632929 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:21:58.632929 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:21:58.632929 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:21:58.632929 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 253/513] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Aug 24 06:21:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:21:58.633623 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e26319-5fae-44d1-bd06-acac38b6ae94 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:58.646309 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25416a65-3f0c-4c8b-8f9e-adbc4f3cf8d4 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:58.665951 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:58.681420 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bcdcc0-ea3c-4936-ab9c-76516a0a4cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:58.682078 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bcdcc0-ea3c-4936-ab9c-76516a0a4cb0 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:58.682356 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bcdcc0-ea3c-4936-ab9c-76516a0a4cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:58.682626 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bcdcc0-ea3c-4936-ab9c-76516a0a4cb0 None admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:58.683808 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bcdcc0-ea3c-4936-ab9c-76516a0a4cb0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:58.684112 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bcdcc0-ea3c-4936-ab9c-76516a0a4cb0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:58.685616 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bcdcc0-ea3c-4936-ab9c-76516a0a4cb0 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:58.697981 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 261/514] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 24 06:21:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:58.699421 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bcdcc0-ea3c-4936-ab9c-76516a0a4cb0 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:58.726060 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:58.748798 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e26319-5fae-44d1-bd06-acac38b6ae94 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:58.787996 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b1c23f-369a-48ef-9489-b084c9719cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:58.788631 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b1c23f-369a-48ef-9489-b084c9719cec None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:58.788908 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b1c23f-369a-48ef-9489-b084c9719cec None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:58.789207 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b1c23f-369a-48ef-9489-b084c9719cec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:58.799685 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b1c23f-369a-48ef-9489-b084c9719cec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:58.801130 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b1c23f-369a-48ef-9489-b084c9719cec None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:58.809739 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bcdcc0-ea3c-4936-ab9c-76516a0a4cb0 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:58.817394 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-09b1c23f-369a-48ef-9489-b084c9719cec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:21:58.817394 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:21:58.817394 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:21:58.817394 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:21:58.817394 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:21:58.817394 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:21:58.817394 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:21:58.817394 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:21:58.817394 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:21:58.817394 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:21:58.817394 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:21:58.817394 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:21:58.817394 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:21:58.817394 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:21:58.817394 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:21:58.817394 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:21:58.818366 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:21:58.818366 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:21:58.818366 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:21:58.818366 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:21:58.818366 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:21:58.818366 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:21:58.818366 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:21:58.818366 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:21:58.818366 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:21:58.818366 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:21:58.818366 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:21:58.818366 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:21:58.818366 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:21:58.818366 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:21:58.818366 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:21:58.818366 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:21:58.818366 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:21:58.819211 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:21:58.819211 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:21:58.819211 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:21:58.819211 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 254/515] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:21:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:21:58.819423 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b1c23f-369a-48ef-9489-b084c9719cec None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:58.851104 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47f49f9-0a92-4ab1-825a-25dde1ad1e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:58.851717 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f49f9-0a92-4ab1-825a-25dde1ad1e4f None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:58.852009 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f49f9-0a92-4ab1-825a-25dde1ad1e4f None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:58.852286 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47f49f9-0a92-4ab1-825a-25dde1ad1e4f None admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:58.853028 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47f49f9-0a92-4ab1-825a-25dde1ad1e4f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:58.853387 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47f49f9-0a92-4ab1-825a-25dde1ad1e4f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:58.855077 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47f49f9-0a92-4ab1-825a-25dde1ad1e4f None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:58.874781 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 262/516] 10.222.0.47 () {56 vars in 1137 bytes} [Wed Aug 24 06:21:58 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:58.875731 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47f49f9-0a92-4ab1-825a-25dde1ad1e4f None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:58.905370 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:58.929299 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b1c23f-369a-48ef-9489-b084c9719cec None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:58.966159 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044ea301-cdda-4c8b-b02e-12df07030763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:58.966805 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044ea301-cdda-4c8b-b02e-12df07030763 None admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:58.967080 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044ea301-cdda-4c8b-b02e-12df07030763 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:58.967354 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044ea301-cdda-4c8b-b02e-12df07030763 None admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:58.967943 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:58.968053 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044ea301-cdda-4c8b-b02e-12df07030763 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:58.969443 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044ea301-cdda-4c8b-b02e-12df07030763 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:58.987316 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47f49f9-0a92-4ab1-825a-25dde1ad1e4f None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:59.024107 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b9f0e4-b563-4a35-810b-cdf6d59785bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:59.024972 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b9f0e4-b563-4a35-810b-cdf6d59785bc None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:59.025306 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b9f0e4-b563-4a35-810b-cdf6d59785bc None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:59.025579 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b9f0e4-b563-4a35-810b-cdf6d59785bc None admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:59.026235 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b9f0e4-b563-4a35-810b-cdf6d59785bc None admin] RBAC: Adding query filter params (name=service) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:59.026534 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b9f0e4-b563-4a35-810b-cdf6d59785bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:59.028019 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b9f0e4-b563-4a35-810b-cdf6d59785bc None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:59.035684 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 263/517] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Aug 24 06:21:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:59.036449 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b9f0e4-b563-4a35-810b-cdf6d59785bc None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:59.054765 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 255/518] 10.222.0.47 () {60 vars in 1132 bytes} [Wed Aug 24 06:21:58 2022] POST /identity/v3/users => generated 247 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:21:59.074295 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044ea301-cdda-4c8b-b02e-12df07030763 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:59.151339 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b9f0e4-b563-4a35-810b-cdf6d59785bc None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:59.192573 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd1c618-7ddb-440e-8dbf-0c80094b6dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:59.193129 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd1c618-7ddb-440e-8dbf-0c80094b6dec None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:59.193437 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd1c618-7ddb-440e-8dbf-0c80094b6dec None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:59.193711 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd1c618-7ddb-440e-8dbf-0c80094b6dec None admin] PATH_INFO: `/v3/projects/service` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:59.194532 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044ea301-cdda-4c8b-b02e-12df07030763 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:59.203077 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd1c618-7ddb-440e-8dbf-0c80094b6dec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:59.204463 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd1c618-7ddb-440e-8dbf-0c80094b6dec None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:59.218496 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-8dd1c618-7ddb-440e-8dbf-0c80094b6dec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:21:59.218496 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:21:59.218496 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:21:59.218496 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:21:59.218496 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:21:59.218496 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:21:59.218496 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:21:59.218496 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:21:59.218496 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:21:59.218496 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:21:59.218496 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:21:59.218496 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:21:59.218496 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 06:21:59.218496 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 06:21:59.218496 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 06:21:59.219502 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 06:21:59.219502 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:21:59.219502 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:21:59.219502 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:21:59.219502 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:21:59.219502 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:21:59.219502 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:21:59.219502 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:21:59.219502 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:21:59.219502 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:21:59.219502 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:21:59.219502 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:21:59.219502 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:21:59.219502 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:21:59.219502 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:21:59.219502 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:21:59.219502 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:21:59.220399 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 06:21:59.220399 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 06:21:59.220399 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:21:59.220399 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:21:59.220399 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:21:59.220399 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:21:59.220399 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:21:59.220399 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:21:59.220399 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 264/519] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 24 06:21:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:21:59.221456 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd1c618-7ddb-440e-8dbf-0c80094b6dec None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:59.226140 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:59.239167 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789bcb54-44b7-40ba-abbe-c5b6719b9316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:59.240023 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789bcb54-44b7-40ba-abbe-c5b6719b9316 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:59.250979 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789bcb54-44b7-40ba-abbe-c5b6719b9316 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:59.250979 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789bcb54-44b7-40ba-abbe-c5b6719b9316 None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:59.250979 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789bcb54-44b7-40ba-abbe-c5b6719b9316 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:59.250979 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789bcb54-44b7-40ba-abbe-c5b6719b9316 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:59.250979 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789bcb54-44b7-40ba-abbe-c5b6719b9316 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:59.250979 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:59.257280 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 256/520] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 24 06:21:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:59.258534 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789bcb54-44b7-40ba-abbe-c5b6719b9316 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:59.349916 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd1c618-7ddb-440e-8dbf-0c80094b6dec None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:59.374247 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789bcb54-44b7-40ba-abbe-c5b6719b9316 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:59.390427 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29519d70-eb37-4907-b703-e857cdc85f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:59.391351 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29519d70-eb37-4907-b703-e857cdc85f74 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:59.391784 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29519d70-eb37-4907-b703-e857cdc85f74 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:59.392200 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29519d70-eb37-4907-b703-e857cdc85f74 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:59.421065 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a5769f-57cd-4e61-848e-70db8b27bc3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:59.421754 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a5769f-57cd-4e61-848e-70db8b27bc3d None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:59.422032 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a5769f-57cd-4e61-848e-70db8b27bc3d None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:59.422300 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a5769f-57cd-4e61-848e-70db8b27bc3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:59.435744 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a5769f-57cd-4e61-848e-70db8b27bc3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:59.437270 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a5769f-57cd-4e61-848e-70db8b27bc3d None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:59.453518 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-12a5769f-57cd-4e61-848e-70db8b27bc3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:21:59.453518 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:21:59.453518 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:21:59.453518 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:21:59.453518 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:21:59.453518 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:21:59.453518 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:21:59.453518 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:21:59.453518 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:21:59.453518 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:21:59.453518 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:21:59.453518 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:21:59.453518 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:21:59.453518 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:21:59.453518 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:21:59.453518 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:21:59.454637 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:21:59.454637 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:21:59.454637 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:21:59.454637 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:21:59.454637 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:21:59.454637 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:21:59.454637 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:21:59.454637 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:21:59.454637 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:21:59.454637 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:21:59.454637 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:21:59.454637 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:21:59.454637 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:21:59.454637 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:21:59.454637 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:21:59.454637 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:21:59.454637 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:21:59.455628 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:21:59.455628 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:21:59.455628 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:21:59.455628 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 257/521] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:21:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:21:59.455842 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a5769f-57cd-4e61-848e-70db8b27bc3d None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:59.479705 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29519d70-eb37-4907-b703-e857cdc85f74 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:59.482069 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29519d70-eb37-4907-b703-e857cdc85f74 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:59.485951 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:59.512728 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:59.581709 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a5769f-57cd-4e61-848e-70db8b27bc3d None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:59.604847 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 265/522] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 24 06:21:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 06:21:59.604847 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29519d70-eb37-4907-b703-e857cdc85f74 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:59.624350 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae18ac9-44bf-443d-bbc7-04936240464e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:59.624594 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae18ac9-44bf-443d-bbc7-04936240464e None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:59.625029 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae18ac9-44bf-443d-bbc7-04936240464e None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:59.625343 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae18ac9-44bf-443d-bbc7-04936240464e None admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:59.626050 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae18ac9-44bf-443d-bbc7-04936240464e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:59.626358 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae18ac9-44bf-443d-bbc7-04936240464e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:59.627832 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae18ac9-44bf-443d-bbc7-04936240464e None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:59.649914 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 258/523] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Aug 24 06:21:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:59.650774 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae18ac9-44bf-443d-bbc7-04936240464e None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:59.725142 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29519d70-eb37-4907-b703-e857cdc85f74 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:59.728878 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:59.765361 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4644c841-f519-4d54-ae15-3bfb5830f285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:59.766002 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4644c841-f519-4d54-ae15-3bfb5830f285 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:59.766503 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4644c841-f519-4d54-ae15-3bfb5830f285 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:59.766820 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4644c841-f519-4d54-ae15-3bfb5830f285 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:59.769688 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae18ac9-44bf-443d-bbc7-04936240464e None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:59.774039 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4644c841-f519-4d54-ae15-3bfb5830f285 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:59.775741 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4644c841-f519-4d54-ae15-3bfb5830f285 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:59.779111 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:21:59.788872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-4644c841-f519-4d54-ae15-3bfb5830f285 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:21:59.788872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:21:59.788872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:21:59.788872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:21:59.788872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:21:59.788872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:21:59.788872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:21:59.788872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:21:59.788872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:21:59.788872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:21:59.788872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:21:59.788872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:21:59.788872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 06:21:59.788872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 06:21:59.788872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 06:21:59.789940 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 06:21:59.789940 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:21:59.789940 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:21:59.789940 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:21:59.789940 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:21:59.789940 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:21:59.789940 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:21:59.789940 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:21:59.789940 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:21:59.789940 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:21:59.789940 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:21:59.789940 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:21:59.789940 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:21:59.789940 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:21:59.789940 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:21:59.789940 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:21:59.789940 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:21:59.790927 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 06:21:59.790927 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 06:21:59.790927 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:21:59.790927 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:21:59.790927 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:21:59.790927 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:21:59.790927 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:21:59.790927 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:21:59.790927 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 266/524] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 24 06:21:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:21:59.790927 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4644c841-f519-4d54-ae15-3bfb5830f285 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:59.820669 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60047b1a-d1b1-45e8-a503-ae1567678c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:59.821664 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60047b1a-d1b1-45e8-a503-ae1567678c97 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:59.824483 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60047b1a-d1b1-45e8-a503-ae1567678c97 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:59.824483 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60047b1a-d1b1-45e8-a503-ae1567678c97 None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:59.824483 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60047b1a-d1b1-45e8-a503-ae1567678c97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:59.824483 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60047b1a-d1b1-45e8-a503-ae1567678c97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:59.826450 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60047b1a-d1b1-45e8-a503-ae1567678c97 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:59.840086 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 259/525] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 24 06:21:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:21:59.841103 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60047b1a-d1b1-45e8-a503-ae1567678c97 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:21:59.924660 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4644c841-f519-4d54-ae15-3bfb5830f285 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:59.952372 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60047b1a-d1b1-45e8-a503-ae1567678c97 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:21:59.969373 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4835169d-4b58-4dba-b5e1-9a76361725a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:59.970280 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4835169d-4b58-4dba-b5e1-9a76361725a3 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:59.970657 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4835169d-4b58-4dba-b5e1-9a76361725a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:59.971151 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4835169d-4b58-4dba-b5e1-9a76361725a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:59.983739 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4835169d-4b58-4dba-b5e1-9a76361725a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:59.985273 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4835169d-4b58-4dba-b5e1-9a76361725a3 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:21:59.988777 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9ea6d6-a486-4710-8091-86d099d0d3a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:21:59.989431 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9ea6d6-a486-4710-8091-86d099d0d3a6 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:21:59.989719 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9ea6d6-a486-4710-8091-86d099d0d3a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:21:59.989995 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9ea6d6-a486-4710-8091-86d099d0d3a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:21:59.990739 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9ea6d6-a486-4710-8091-86d099d0d3a6 None admin] RBAC: Adding query filter params (role.id=1712d85aedaf4189bf4bc89094d888c3, scope.project.id=c02e49a89cdd40d885bfa816f06c58a1, user.id=1f4f49c1b08c460f8ba6c451c9c92c2a) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:21:59.991058 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9ea6d6-a486-4710-8091-86d099d0d3a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:21:59.992485 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9ea6d6-a486-4710-8091-86d099d0d3a6 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:00.008053 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 260/526] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 24 06:21:59 2022] GET /identity/v3/role_assignments?user.id=1f4f49c1b08c460f8ba6c451c9c92c2a&scope.project.id=c02e49a89cdd40d885bfa816f06c58a1&role.id=1712d85aedaf4189bf4bc89094d888c3 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:00.008053 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9ea6d6-a486-4710-8091-86d099d0d3a6 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:00.009825 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-4835169d-4b58-4dba-b5e1-9a76361725a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:00.009825 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:00.009825 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:00.009825 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:00.009825 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:00.009825 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:00.009825 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:00.009825 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:00.009825 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:00.009825 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:00.009825 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:00.009825 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:00.009825 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:22:00.009825 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:22:00.009825 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:22:00.009825 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:22:00.011050 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:00.011050 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:00.011050 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:00.011050 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:00.011050 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:00.011050 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:00.011050 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:00.011050 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:00.011050 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:00.011050 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:00.011050 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:00.011050 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:00.011050 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:00.011050 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:00.011050 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:00.011050 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:00.011050 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:22:00.012113 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:22:00.012113 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:00.012113 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:00.013251 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 267/527] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:21:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:00.014272 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4835169d-4b58-4dba-b5e1-9a76361725a3 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:00.038936 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:00.070967 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:00.143180 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4835169d-4b58-4dba-b5e1-9a76361725a3 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:00.181370 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9ea6d6-a486-4710-8091-86d099d0d3a6 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:00.195806 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa7d2b5-76aa-4020-810b-73d329b11a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:00.196876 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa7d2b5-76aa-4020-810b-73d329b11a1f None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:00.197171 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa7d2b5-76aa-4020-810b-73d329b11a1f None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:00.197476 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa7d2b5-76aa-4020-810b-73d329b11a1f None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:00.209061 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa7d2b5-76aa-4020-810b-73d329b11a1f None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:00.211000 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa7d2b5-76aa-4020-810b-73d329b11a1f None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:00.227804 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1408df97-f991-4791-8860-eb6eb5ce21b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:00.228687 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1408df97-f991-4791-8860-eb6eb5ce21b4 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:00.229161 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1408df97-f991-4791-8860-eb6eb5ce21b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:00.229296 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-cfa7d2b5-76aa-4020-810b-73d329b11a1f None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 06:22:00.229296 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:00.229296 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:00.229296 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:00.229296 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:00.229296 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:00.229296 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:00.229296 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:00.229296 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:00.229296 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:00.229296 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:00.229296 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:00.229296 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 06:22:00.229296 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 06:22:00.229296 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 06:22:00.229296 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 06:22:00.231161 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:00.231161 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:00.231161 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 06:22:00.231161 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:22:00.231161 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 06:22:00.231161 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:22:00.231161 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:00.231161 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:00.231161 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:00.231161 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:00.231161 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:00.231161 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:00.231161 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:00.231161 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:00.231161 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:00.231161 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:00.231161 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:00.231161 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:00.232104 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:00.232104 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:00.232104 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 06:22:00.232104 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 06:22:00.232104 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 06:22:00.232104 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 06:22:00.232104 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 06:22:00.232104 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 06:22:00.232104 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 06:22:00.232104 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:00.232104 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 268/528] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Aug 24 06:22:00 2022] GET /identity/v3/users/neutron => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:00.232853 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa7d2b5-76aa-4020-810b-73d329b11a1f None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:00.233333 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1408df97-f991-4791-8860-eb6eb5ce21b4 None admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:00.234344 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1408df97-f991-4791-8860-eb6eb5ce21b4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:00.234804 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1408df97-f991-4791-8860-eb6eb5ce21b4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:00.236863 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1408df97-f991-4791-8860-eb6eb5ce21b4 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:00.250844 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 261/529] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Aug 24 06:21:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:00.252080 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1408df97-f991-4791-8860-eb6eb5ce21b4 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:00.343843 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:00.377320 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa7d2b5-76aa-4020-810b-73d329b11a1f None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:00.378724 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:00.390335 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1408df97-f991-4791-8860-eb6eb5ce21b4 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:00.424561 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1889bf-b695-42ef-a7ad-e95e46d88a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:00.425390 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1889bf-b695-42ef-a7ad-e95e46d88a02 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:00.425678 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1889bf-b695-42ef-a7ad-e95e46d88a02 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:00.425951 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1889bf-b695-42ef-a7ad-e95e46d88a02 None admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:00.426957 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1889bf-b695-42ef-a7ad-e95e46d88a02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:00.427505 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1889bf-b695-42ef-a7ad-e95e46d88a02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:00.428920 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1889bf-b695-42ef-a7ad-e95e46d88a02 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:00.431802 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961d422c-3085-402c-8e11-e5770b3601cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:00.432918 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961d422c-3085-402c-8e11-e5770b3601cb None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:00.433522 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961d422c-3085-402c-8e11-e5770b3601cb None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:00.433945 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961d422c-3085-402c-8e11-e5770b3601cb None admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:00.434919 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961d422c-3085-402c-8e11-e5770b3601cb None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:00.435400 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961d422c-3085-402c-8e11-e5770b3601cb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:00.437113 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961d422c-3085-402c-8e11-e5770b3601cb None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:00.444483 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 269/530] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 24 06:22:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:00.445698 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1889bf-b695-42ef-a7ad-e95e46d88a02 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:00.460997 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 262/531] 10.222.0.47 () {56 vars in 1139 bytes} [Wed Aug 24 06:22:00 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:00.462167 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961d422c-3085-402c-8e11-e5770b3601cb None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:00.580490 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961d422c-3085-402c-8e11-e5770b3601cb None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:00.586216 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1889bf-b695-42ef-a7ad-e95e46d88a02 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:00.629335 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad8afbd-0bdf-487f-96b6-664e95bfa4ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:00.630686 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad8afbd-0bdf-487f-96b6-664e95bfa4ee None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:00.631191 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad8afbd-0bdf-487f-96b6-664e95bfa4ee None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:00.631609 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d796d12-0179-4396-8dd9-474beedeb4c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:00.632036 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad8afbd-0bdf-487f-96b6-664e95bfa4ee None admin] PATH_INFO: `/v3/projects/service` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:00.632418 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d796d12-0179-4396-8dd9-474beedeb4c6 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:00.632786 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d796d12-0179-4396-8dd9-474beedeb4c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:00.633210 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d796d12-0179-4396-8dd9-474beedeb4c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:00.634026 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d796d12-0179-4396-8dd9-474beedeb4c6 None admin] RBAC: Adding query filter params (role.id=1712d85aedaf4189bf4bc89094d888c3, scope.project.id=c02e49a89cdd40d885bfa816f06c58a1, user.id=36639dce4f97412f89d71367f650b901) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:00.634431 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d796d12-0179-4396-8dd9-474beedeb4c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:00.636262 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d796d12-0179-4396-8dd9-474beedeb4c6 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:00.645582 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:00.647863 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad8afbd-0bdf-487f-96b6-664e95bfa4ee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:00.649817 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad8afbd-0bdf-487f-96b6-664e95bfa4ee None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:00.649938 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 270/532] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 24 06:22:00 2022] GET /identity/v3/role_assignments?user.id=36639dce4f97412f89d71367f650b901&scope.project.id=c02e49a89cdd40d885bfa816f06c58a1&role.id=1712d85aedaf4189bf4bc89094d888c3 => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:00.650940 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d796d12-0179-4396-8dd9-474beedeb4c6 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:00.664912 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-dad8afbd-0bdf-487f-96b6-664e95bfa4ee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:22:00.664912 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:00.664912 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:00.664912 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:00.664912 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:00.664912 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:00.664912 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:00.664912 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:00.664912 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:00.664912 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:00.664912 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:00.664912 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:00.664912 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 06:22:00.664912 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 06:22:00.664912 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 06:22:00.666121 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 06:22:00.666121 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:00.666121 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:00.666121 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:00.666121 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:00.666121 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:00.666121 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:00.666121 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:00.666121 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:00.666121 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:00.666121 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:00.666121 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:00.666121 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:00.666121 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:00.666121 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:00.666121 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:00.666121 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:00.667627 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 06:22:00.667627 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 06:22:00.667627 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:22:00.667627 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:22:00.667627 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:22:00.667627 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:22:00.667627 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:22:00.667627 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:00.669657 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 263/533] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 24 06:22:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:00.678933 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad8afbd-0bdf-487f-96b6-664e95bfa4ee None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:00.702151 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:00.878789 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d796d12-0179-4396-8dd9-474beedeb4c6 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:00.884049 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad8afbd-0bdf-487f-96b6-664e95bfa4ee None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:00.931920 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6057df7-ebf0-4924-98c9-d88a1888f038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:00.933402 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6057df7-ebf0-4924-98c9-d88a1888f038 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:00.934025 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6057df7-ebf0-4924-98c9-d88a1888f038 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:00.934508 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6057df7-ebf0-4924-98c9-d88a1888f038 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:00.965226 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bbdee6-efa9-4ccd-877e-7d8a1393a692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:00.966830 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bbdee6-efa9-4ccd-877e-7d8a1393a692 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:00.967993 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bbdee6-efa9-4ccd-877e-7d8a1393a692 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:00.968513 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bbdee6-efa9-4ccd-877e-7d8a1393a692 None admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:00.969413 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bbdee6-efa9-4ccd-877e-7d8a1393a692 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:00.970665 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bbdee6-efa9-4ccd-877e-7d8a1393a692 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:00.972973 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bbdee6-efa9-4ccd-877e-7d8a1393a692 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:01.000887 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 264/534] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Aug 24 06:22:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:01.008218 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bbdee6-efa9-4ccd-877e-7d8a1393a692 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:01.066898 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:01.070178 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6057df7-ebf0-4924-98c9-d88a1888f038 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:01.071680 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6057df7-ebf0-4924-98c9-d88a1888f038 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:01.102904 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:01.143969 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bbdee6-efa9-4ccd-877e-7d8a1393a692 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:01.187436 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb2e22a-0e42-4ed8-b0d6-9a06e3f89bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:01.188717 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb2e22a-0e42-4ed8-b0d6-9a06e3f89bf4 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:01.189175 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb2e22a-0e42-4ed8-b0d6-9a06e3f89bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:01.189808 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb2e22a-0e42-4ed8-b0d6-9a06e3f89bf4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:01.191084 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb2e22a-0e42-4ed8-b0d6-9a06e3f89bf4 None admin] RBAC: Adding query filter params (role.id=1712d85aedaf4189bf4bc89094d888c3, scope.project.id=c02e49a89cdd40d885bfa816f06c58a1, user.id=31b331b6ddfe46aeae7b52fcbbc27771) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:01.191827 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb2e22a-0e42-4ed8-b0d6-9a06e3f89bf4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:01.193991 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb2e22a-0e42-4ed8-b0d6-9a06e3f89bf4 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:01.203356 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 265/535] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 24 06:22:01 2022] GET /identity/v3/role_assignments?user.id=31b331b6ddfe46aeae7b52fcbbc27771&scope.project.id=c02e49a89cdd40d885bfa816f06c58a1&role.id=1712d85aedaf4189bf4bc89094d888c3 => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:22:01.205904 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2edc02d-0780-4967-bd6f-2c93bf6e415b None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:01.206501 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2edc02d-0780-4967-bd6f-2c93bf6e415b None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:01.207082 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2edc02d-0780-4967-bd6f-2c93bf6e415b None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:01.210370 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 266/536] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:22:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:22:01.218946 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1760aed6-0ceb-4782-ab89-245bd8539ec0 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:01.219254 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1760aed6-0ceb-4782-ab89-245bd8539ec0 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:01.219670 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1760aed6-0ceb-4782-ab89-245bd8539ec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:01.239884 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-1760aed6-0ceb-4782-ab89-245bd8539ec0 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:22:01.241248 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-1760aed6-0ceb-4782-ab89-245bd8539ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:22:01.251296 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 271/537] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 24 06:22:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 601 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 06:22:01.259133 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6057df7-ebf0-4924-98c9-d88a1888f038 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:01.365153 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 267/538] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:22:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:22:01.416448 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:01.425501 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6057df7-ebf0-4924-98c9-d88a1888f038 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:01.480221 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596b21d1-a976-4b1c-9e76-c1222d1daf7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:01.481238 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596b21d1-a976-4b1c-9e76-c1222d1daf7a None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:01.481614 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596b21d1-a976-4b1c-9e76-c1222d1daf7a None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:01.481981 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596b21d1-a976-4b1c-9e76-c1222d1daf7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:01.501213 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596b21d1-a976-4b1c-9e76-c1222d1daf7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:01.502839 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596b21d1-a976-4b1c-9e76-c1222d1daf7a None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:01.521836 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-596b21d1-a976-4b1c-9e76-c1222d1daf7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:01.521836 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:01.521836 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:01.521836 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:01.521836 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:01.521836 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:01.521836 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:01.521836 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:01.521836 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:01.521836 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:01.521836 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:01.521836 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:01.521836 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:22:01.521836 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:22:01.521836 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:22:01.521836 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:22:01.522985 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:01.522985 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:01.522985 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:01.522985 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:01.522985 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:01.522985 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:01.522985 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:01.522985 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:01.522985 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:01.522985 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:01.522985 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:01.522985 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:01.522985 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:01.522985 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:01.522985 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:01.522985 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:01.522985 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:22:01.524016 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:22:01.524016 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:01.524016 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:01.525341 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 272/539] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:22:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:01.531878 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1760aed6-0ceb-4782-ab89-245bd8539ec0 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:01.641608 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:01.701825 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1760aed6-0ceb-4782-ab89-245bd8539ec0 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:01.734955 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596b21d1-a976-4b1c-9e76-c1222d1daf7a None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:01.752581 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0359c9-579e-46c5-b237-4d77759f15cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:01.753223 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0359c9-579e-46c5-b237-4d77759f15cc None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:01.753505 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0359c9-579e-46c5-b237-4d77759f15cc None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:01.753779 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0359c9-579e-46c5-b237-4d77759f15cc None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:01.754437 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0359c9-579e-46c5-b237-4d77759f15cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:01.754739 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0359c9-579e-46c5-b237-4d77759f15cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:01.756124 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0359c9-579e-46c5-b237-4d77759f15cc None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:01.788959 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 268/540] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 24 06:22:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:01.815227 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0359c9-579e-46c5-b237-4d77759f15cc None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:01.987384 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0359c9-579e-46c5-b237-4d77759f15cc None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:01.990688 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596b21d1-a976-4b1c-9e76-c1222d1daf7a None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:02.033804 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:02.049538 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734c1e8a-9cb1-4048-9dd8-274b445e1992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:02.050191 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734c1e8a-9cb1-4048-9dd8-274b445e1992 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:02.050465 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734c1e8a-9cb1-4048-9dd8-274b445e1992 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:02.054224 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734c1e8a-9cb1-4048-9dd8-274b445e1992 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:02.059157 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9ae1bc-590d-47f4-ac6b-f03bfcb03b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:02.059901 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9ae1bc-590d-47f4-ac6b-f03bfcb03b7c None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:02.060022 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734c1e8a-9cb1-4048-9dd8-274b445e1992 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:02.060206 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9ae1bc-590d-47f4-ac6b-f03bfcb03b7c None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:02.060451 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9ae1bc-590d-47f4-ac6b-f03bfcb03b7c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:02.061485 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734c1e8a-9cb1-4048-9dd8-274b445e1992 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:02.074117 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-734c1e8a-9cb1-4048-9dd8-274b445e1992 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:22:02.074117 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:02.074117 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:02.074117 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:02.074117 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:02.074117 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:02.074117 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:02.074117 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:02.074117 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:02.074117 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:02.074117 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:02.074117 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:02.074117 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 06:22:02.074117 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 06:22:02.074117 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 06:22:02.075040 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 06:22:02.075040 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:02.075040 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:02.075040 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:02.075040 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:02.075040 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:02.075040 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:02.075040 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:02.075040 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:02.075040 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:02.075040 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:02.075040 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:02.075040 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:02.075040 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:02.075040 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:02.075040 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:02.075040 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:02.075871 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 06:22:02.075871 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 06:22:02.075871 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:22:02.075871 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:22:02.075871 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:22:02.075871 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:22:02.075871 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:22:02.075871 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:02.079947 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 269/541] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 24 06:22:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:02.081219 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734c1e8a-9cb1-4048-9dd8-274b445e1992 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:02.094677 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9ae1bc-590d-47f4-ac6b-f03bfcb03b7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:22:02.102870 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-7c9ae1bc-590d-47f4-ac6b-f03bfcb03b7c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:22:02.102870 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:02.102870 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:02.102870 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:02.102870 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:02.102870 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:02.102870 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:02.102870 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:02.102870 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:02.102870 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:02.102870 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:02.102870 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:02.102870 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:22:02.102870 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:22:02.102870 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:22:02.102870 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:22:02.103797 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:22:02.103797 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:22:02.103797 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:22:02.103797 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:22:02.103797 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:02.103797 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:02.103797 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:02.103797 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:02.103797 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:02.103797 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:02.103797 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:02.103797 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:02.103797 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:02.103797 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:02.103797 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:02.103797 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:02.105424 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:02.105424 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:02.105424 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:02.105424 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:02.105424 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:22:02.105424 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:22:02.105424 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:22:02.105424 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:22:02.105424 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:22:02.105424 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:22:02.105424 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:22:02.105424 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:02.113840 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 273/542] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Aug 24 06:22:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 377 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:02.122304 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9ae1bc-590d-47f4-ac6b-f03bfcb03b7c None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:02.159084 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:02.196605 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734c1e8a-9cb1-4048-9dd8-274b445e1992 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:02.237226 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522d5077-cb5d-4ae6-afbf-4c9f22da4f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:02.237226 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d5077-cb5d-4ae6-afbf-4c9f22da4f81 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:02.237226 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d5077-cb5d-4ae6-afbf-4c9f22da4f81 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:02.237226 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522d5077-cb5d-4ae6-afbf-4c9f22da4f81 None admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:02.237654 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522d5077-cb5d-4ae6-afbf-4c9f22da4f81 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:02.237654 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522d5077-cb5d-4ae6-afbf-4c9f22da4f81 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:02.238578 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522d5077-cb5d-4ae6-afbf-4c9f22da4f81 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:02.256030 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 270/543] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Aug 24 06:22:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:22:02.258205 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522d5077-cb5d-4ae6-afbf-4c9f22da4f81 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:02.306545 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:02.361119 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9ae1bc-590d-47f4-ac6b-f03bfcb03b7c None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:02.377802 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522d5077-cb5d-4ae6-afbf-4c9f22da4f81 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:02.399620 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c13d59f-28f2-4871-a9ea-9e85e1413743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:02.400819 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c13d59f-28f2-4871-a9ea-9e85e1413743 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:02.401656 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c13d59f-28f2-4871-a9ea-9e85e1413743 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:02.402565 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c13d59f-28f2-4871-a9ea-9e85e1413743 None admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:02.403505 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c13d59f-28f2-4871-a9ea-9e85e1413743 None admin] RBAC: Adding query filter params (name=service) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:02.404028 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c13d59f-28f2-4871-a9ea-9e85e1413743 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:02.405672 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c13d59f-28f2-4871-a9ea-9e85e1413743 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:02.414855 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 274/544] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Aug 24 06:22:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:02.416298 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fadbcf-f945-4d33-b7a9-54ff5240a306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:02.416597 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53423fd1-9fe0-44ed-bb32-a8c07a5d3a98 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:02.416909 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fadbcf-f945-4d33-b7a9-54ff5240a306 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:02.417330 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fadbcf-f945-4d33-b7a9-54ff5240a306 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:02.417604 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fadbcf-f945-4d33-b7a9-54ff5240a306 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:02.417817 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53423fd1-9fe0-44ed-bb32-a8c07a5d3a98 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:02.418284 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53423fd1-9fe0-44ed-bb32-a8c07a5d3a98 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:02.418392 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fadbcf-f945-4d33-b7a9-54ff5240a306 None admin] RBAC: Adding query filter params (role.id=1712d85aedaf4189bf4bc89094d888c3, scope.project.id=c02e49a89cdd40d885bfa816f06c58a1, user.id=f0ba3f03ec15475ab6ebf64994ad742c) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:02.418642 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fadbcf-f945-4d33-b7a9-54ff5240a306 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:02.420233 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fadbcf-f945-4d33-b7a9-54ff5240a306 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:02.424741 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 275/545] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:22:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:22:02.425896 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53423fd1-9fe0-44ed-bb32-a8c07a5d3a98 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:02.428381 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 271/546] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 24 06:22:02 2022] GET /identity/v3/role_assignments?user.id=f0ba3f03ec15475ab6ebf64994ad742c&scope.project.id=c02e49a89cdd40d885bfa816f06c58a1&role.id=1712d85aedaf4189bf4bc89094d888c3 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:22:02.428381 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d3bb0-c92c-40bc-939d-c1e6ec403464 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:02.428381 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d3bb0-c92c-40bc-939d-c1e6ec403464 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:02.428649 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d3bb0-c92c-40bc-939d-c1e6ec403464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:02.448231 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-924d3bb0-c92c-40bc-939d-c1e6ec403464 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:22:02.450206 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-924d3bb0-c92c-40bc-939d-c1e6ec403464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:22:02.533392 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 272/547] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:22:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:22:02.561956 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:02.607114 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53423fd1-9fe0-44ed-bb32-a8c07a5d3a98 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:02.663213 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ea24df-fd5d-4d54-b57c-a106cd91e867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:02.663881 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ea24df-fd5d-4d54-b57c-a106cd91e867 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:02.664169 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ea24df-fd5d-4d54-b57c-a106cd91e867 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:02.664451 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ea24df-fd5d-4d54-b57c-a106cd91e867 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:02.676695 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ea24df-fd5d-4d54-b57c-a106cd91e867 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:02.678216 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ea24df-fd5d-4d54-b57c-a106cd91e867 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:02.696554 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-76ea24df-fd5d-4d54-b57c-a106cd91e867 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:02.696554 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:02.696554 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:02.696554 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:02.696554 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:02.696554 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:02.696554 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:02.696554 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:02.696554 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:02.696554 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:02.696554 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:02.696554 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:02.696554 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:22:02.696554 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:22:02.696554 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:22:02.696554 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:22:02.697655 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:02.697655 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:02.697655 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:02.697655 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:02.697655 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:02.697655 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:02.697655 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:02.697655 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:02.697655 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:02.697655 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:02.697655 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:02.697655 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:02.697655 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:02.697655 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:02.697655 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:02.697655 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:02.697655 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:22:02.699503 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:22:02.699503 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:02.699503 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:02.699503 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 276/548] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:22:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:02.704194 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924d3bb0-c92c-40bc-939d-c1e6ec403464 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:02.779203 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:02.835119 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ea24df-fd5d-4d54-b57c-a106cd91e867 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:02.874024 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924d3bb0-c92c-40bc-939d-c1e6ec403464 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:02.930769 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe0c8c6-70e8-4c52-8e6a-8a17be4d2a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:02.931381 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe0c8c6-70e8-4c52-8e6a-8a17be4d2a44 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:02.931659 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe0c8c6-70e8-4c52-8e6a-8a17be4d2a44 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:02.931929 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe0c8c6-70e8-4c52-8e6a-8a17be4d2a44 None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:02.932615 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe0c8c6-70e8-4c52-8e6a-8a17be4d2a44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:02.932908 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe0c8c6-70e8-4c52-8e6a-8a17be4d2a44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:02.934481 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe0c8c6-70e8-4c52-8e6a-8a17be4d2a44 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:02.946705 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 273/549] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 24 06:22:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:02.952962 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b12ec7-56c2-40f9-9751-d459aaeb2518 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:02.953263 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b12ec7-56c2-40f9-9751-d459aaeb2518 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:02.953554 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b12ec7-56c2-40f9-9751-d459aaeb2518 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:02.954894 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 274/550] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:22:02 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:22:02.955634 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b12ec7-56c2-40f9-9751-d459aaeb2518 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:02.960794 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ea24df-fd5d-4d54-b57c-a106cd91e867 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:02.995462 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:03.004824 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebfdf07-f757-4c16-8321-07badfe4c382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:03.005505 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebfdf07-f757-4c16-8321-07badfe4c382 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:03.005816 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebfdf07-f757-4c16-8321-07badfe4c382 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:03.006094 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebfdf07-f757-4c16-8321-07badfe4c382 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:03.018527 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebfdf07-f757-4c16-8321-07badfe4c382 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:22:03.023556 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-bebfdf07-f757-4c16-8321-07badfe4c382 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:22:03.023556 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:03.023556 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:03.023556 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:03.023556 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:03.023556 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:03.023556 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:03.023556 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:03.023556 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:03.023556 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:03.023556 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:03.023556 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:03.023556 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:22:03.023556 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:22:03.023556 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:22:03.023556 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:22:03.024997 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:22:03.024997 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:22:03.024997 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:22:03.024997 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:22:03.024997 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:03.024997 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:03.024997 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:03.024997 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:03.024997 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:03.024997 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:03.024997 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:03.024997 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:03.024997 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:03.024997 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:03.024997 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:03.024997 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:03.025922 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:03.025922 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:03.025922 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:03.025922 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:03.025922 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:22:03.025922 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:22:03.025922 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:22:03.025922 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:22:03.025922 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:22:03.025922 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:22:03.025922 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:22:03.025922 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:03.025922 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 277/551] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Aug 24 06:22:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:03.030447 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adfc46c-a74e-4466-ad35-eb81a4bd03bd None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:03.030625 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adfc46c-a74e-4466-ad35-eb81a4bd03bd None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:03.030897 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adfc46c-a74e-4466-ad35-eb81a4bd03bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:03.052866 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-1adfc46c-a74e-4466-ad35-eb81a4bd03bd None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:22:03.054191 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-1adfc46c-a74e-4466-ad35-eb81a4bd03bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:22:03.064479 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:03.071552 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b12ec7-56c2-40f9-9751-d459aaeb2518 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:03.108184 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4e6e27-ce8e-4404-b531-91fe384e1f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:03.108745 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4e6e27-ce8e-4404-b531-91fe384e1f92 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:03.109018 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4e6e27-ce8e-4404-b531-91fe384e1f92 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:03.109319 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4e6e27-ce8e-4404-b531-91fe384e1f92 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:03.116915 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4e6e27-ce8e-4404-b531-91fe384e1f92 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:03.118383 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4e6e27-ce8e-4404-b531-91fe384e1f92 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:03.131011 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-0e4e6e27-ce8e-4404-b531-91fe384e1f92 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 06:22:03.131011 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:03.131011 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:03.131011 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:03.131011 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:03.131011 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:03.131011 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:03.131011 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:03.131011 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:03.131011 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:03.131011 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:03.131011 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:03.131011 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 06:22:03.131011 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 06:22:03.131011 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 06:22:03.131011 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 06:22:03.133393 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:03.133393 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:03.133393 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 06:22:03.133393 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:22:03.133393 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 06:22:03.133393 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:22:03.133393 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:03.133393 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:03.133393 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:03.133393 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:03.133393 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:03.133393 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:03.133393 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:03.133393 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:03.133393 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:03.133393 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:03.133393 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:03.133393 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:03.136609 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:03.136609 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:03.136609 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 06:22:03.136609 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 06:22:03.136609 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 06:22:03.136609 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 06:22:03.136609 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 06:22:03.136609 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 06:22:03.136609 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 06:22:03.136609 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:03.138049 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 278/552] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:22:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:22:03.138049 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adfc46c-a74e-4466-ad35-eb81a4bd03bd None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:03.138333 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 275/553] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Aug 24 06:22:02 2022] GET /identity/v3/users/swift => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:03.143981 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4e6e27-ce8e-4404-b531-91fe384e1f92 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:03.252644 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adfc46c-a74e-4466-ad35-eb81a4bd03bd None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:03.272249 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:03.293931 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4809c1-5503-4adf-b15d-af4862e562c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:03.294539 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4809c1-5503-4adf-b15d-af4862e562c6 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:03.294818 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4809c1-5503-4adf-b15d-af4862e562c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:03.295088 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4809c1-5503-4adf-b15d-af4862e562c6 None admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:03.295758 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4809c1-5503-4adf-b15d-af4862e562c6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:03.296078 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4809c1-5503-4adf-b15d-af4862e562c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:03.297635 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4809c1-5503-4adf-b15d-af4862e562c6 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:03.309723 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4e6e27-ce8e-4404-b531-91fe384e1f92 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:03.314240 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 279/554] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Aug 24 06:22:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:03.319509 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4809c1-5503-4adf-b15d-af4862e562c6 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:03.350066 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aaf33d-9ea4-4c73-9ef0-7f3f3f14a6c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:03.351617 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aaf33d-9ea4-4c73-9ef0-7f3f3f14a6c5 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:03.352106 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aaf33d-9ea4-4c73-9ef0-7f3f3f14a6c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:03.352584 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aaf33d-9ea4-4c73-9ef0-7f3f3f14a6c5 None admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:03.353592 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3aaf33d-9ea4-4c73-9ef0-7f3f3f14a6c5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:03.354400 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3aaf33d-9ea4-4c73-9ef0-7f3f3f14a6c5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:03.356438 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3aaf33d-9ea4-4c73-9ef0-7f3f3f14a6c5 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:03.377654 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 276/555] 10.222.0.47 () {56 vars in 1135 bytes} [Wed Aug 24 06:22:03 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:22:03.384384 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aaf33d-9ea4-4c73-9ef0-7f3f3f14a6c5 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:03.409566 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:03.509542 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aaf33d-9ea4-4c73-9ef0-7f3f3f14a6c5 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:03.530068 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4809c1-5503-4adf-b15d-af4862e562c6 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:03.546858 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82514312-4158-4089-88a6-afc5f49ff9e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:03.547476 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82514312-4158-4089-88a6-afc5f49ff9e6 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:03.547756 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82514312-4158-4089-88a6-afc5f49ff9e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:03.548026 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82514312-4158-4089-88a6-afc5f49ff9e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:03.580295 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5015bf2-ebee-4c65-9e8f-c2ebd8a1c932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:03.581639 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5015bf2-ebee-4c65-9e8f-c2ebd8a1c932 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:03.582591 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5015bf2-ebee-4c65-9e8f-c2ebd8a1c932 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:03.583354 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5015bf2-ebee-4c65-9e8f-c2ebd8a1c932 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:03.599107 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5015bf2-ebee-4c65-9e8f-c2ebd8a1c932 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:03.601578 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5015bf2-ebee-4c65-9e8f-c2ebd8a1c932 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:03.623345 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82514312-4158-4089-88a6-afc5f49ff9e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:03.624470 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82514312-4158-4089-88a6-afc5f49ff9e6 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:03.624957 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-c5015bf2-ebee-4c65-9e8f-c2ebd8a1c932 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:03.624957 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:03.624957 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:03.624957 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:03.624957 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:03.624957 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:03.624957 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:03.624957 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:03.624957 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:03.624957 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:03.624957 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:03.624957 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:03.624957 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:22:03.624957 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:22:03.624957 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:22:03.624957 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:22:03.625969 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:03.625969 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:03.625969 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:03.625969 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:03.625969 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:03.625969 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:03.625969 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:03.625969 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:03.625969 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:03.625969 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:03.625969 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:03.625969 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:03.625969 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:03.625969 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:03.625969 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:03.625969 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:03.625969 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:22:03.626875 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:22:03.626875 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:03.626875 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:03.635306 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 280/556] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:22:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:22:03.637202 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5015bf2-ebee-4c65-9e8f-c2ebd8a1c932 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:03.642902 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:03.681465 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:03.738968 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 277/557] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 24 06:22:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 06:22:03.738968 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82514312-4158-4089-88a6-afc5f49ff9e6 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:03.765663 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5015bf2-ebee-4c65-9e8f-c2ebd8a1c932 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:03.803931 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb50497-0bd7-4a7d-859b-c18fb95ad045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:03.804513 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb50497-0bd7-4a7d-859b-c18fb95ad045 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:03.804794 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb50497-0bd7-4a7d-859b-c18fb95ad045 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:03.805196 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb50497-0bd7-4a7d-859b-c18fb95ad045 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:03.817089 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb50497-0bd7-4a7d-859b-c18fb95ad045 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:22:03.822196 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-fcb50497-0bd7-4a7d-859b-c18fb95ad045 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:22:03.822196 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:03.822196 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:03.822196 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:03.822196 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:03.822196 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:03.822196 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:03.822196 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:03.822196 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:03.822196 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:03.822196 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:03.822196 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:03.822196 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:22:03.822196 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:22:03.822196 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:22:03.822196 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:22:03.823588 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:22:03.823588 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:22:03.823588 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:22:03.823588 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:22:03.823588 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:03.823588 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:03.823588 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:03.823588 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:03.823588 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:03.823588 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:03.823588 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:03.823588 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:03.823588 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:03.823588 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:03.823588 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:03.823588 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:03.824524 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:03.824524 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:03.824524 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:03.824524 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:03.824524 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:22:03.824524 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:22:03.824524 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:22:03.824524 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:22:03.824524 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:22:03.824524 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:22:03.824524 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:22:03.824524 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:03.824524 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 281/558] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Aug 24 06:22:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:03.825710 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b707e52f-a028-4c2b-8781-c981160a8bd2 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:03.825710 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b707e52f-a028-4c2b-8781-c981160a8bd2 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:03.825710 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b707e52f-a028-4c2b-8781-c981160a8bd2 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:03.829222 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 282/559] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:22:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:22:03.829222 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b707e52f-a028-4c2b-8781-c981160a8bd2 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:03.861375 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82514312-4158-4089-88a6-afc5f49ff9e6 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:03.879753 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:03.902239 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648d6688-aec3-4d1f-8a63-e5acd51779d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:03.902855 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648d6688-aec3-4d1f-8a63-e5acd51779d8 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:03.903144 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648d6688-aec3-4d1f-8a63-e5acd51779d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:03.903419 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648d6688-aec3-4d1f-8a63-e5acd51779d8 None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:03.904061 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648d6688-aec3-4d1f-8a63-e5acd51779d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:03.904519 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648d6688-aec3-4d1f-8a63-e5acd51779d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:03.906014 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648d6688-aec3-4d1f-8a63-e5acd51779d8 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:03.924212 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 278/560] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 24 06:22:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:03.925089 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648d6688-aec3-4d1f-8a63-e5acd51779d8 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:03.932290 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:03.945672 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b707e52f-a028-4c2b-8781-c981160a8bd2 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:03.983267 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66513c50-53c7-43a6-91fd-b46c8cce804a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:03.983897 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66513c50-53c7-43a6-91fd-b46c8cce804a None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:03.984175 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66513c50-53c7-43a6-91fd-b46c8cce804a None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:03.984452 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66513c50-53c7-43a6-91fd-b46c8cce804a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:03.995845 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66513c50-53c7-43a6-91fd-b46c8cce804a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:03.997288 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66513c50-53c7-43a6-91fd-b46c8cce804a None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:04.012622 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-66513c50-53c7-43a6-91fd-b46c8cce804a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:04.012622 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:04.012622 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:04.012622 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:04.012622 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:04.012622 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:04.012622 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:04.012622 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:04.012622 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:04.012622 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:04.012622 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:04.012622 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:04.012622 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:22:04.012622 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:22:04.012622 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:22:04.012622 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:22:04.013798 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:04.013798 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:04.013798 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:04.013798 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:04.013798 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:04.013798 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:04.013798 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:04.013798 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:04.013798 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:04.013798 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:04.013798 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:04.013798 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:04.013798 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:04.013798 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:04.013798 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:04.013798 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:04.013798 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:22:04.019659 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:22:04.019659 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:04.019659 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:04.019659 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 283/561] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:22:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:04.019659 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be74c33-e0a3-466a-8cae-d3d3c9072e0d None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:04.019659 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be74c33-e0a3-466a-8cae-d3d3c9072e0d None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:04.019659 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be74c33-e0a3-466a-8cae-d3d3c9072e0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:04.035723 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-7be74c33-e0a3-466a-8cae-d3d3c9072e0d None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:22:04.036927 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-7be74c33-e0a3-466a-8cae-d3d3c9072e0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:22:04.043177 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648d6688-aec3-4d1f-8a63-e5acd51779d8 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:04.079872 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e5630d-20e0-4b72-9f72-a76780c37882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:04.080488 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e5630d-20e0-4b72-9f72-a76780c37882 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:04.080765 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e5630d-20e0-4b72-9f72-a76780c37882 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:04.081046 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e5630d-20e0-4b72-9f72-a76780c37882 None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:04.081747 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e5630d-20e0-4b72-9f72-a76780c37882 None admin] RBAC: Adding query filter params (name=service) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:04.082046 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e5630d-20e0-4b72-9f72-a76780c37882 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:04.083420 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e5630d-20e0-4b72-9f72-a76780c37882 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:04.096412 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 279/562] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Aug 24 06:22:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:04.096412 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e5630d-20e0-4b72-9f72-a76780c37882 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:04.133218 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 284/563] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:22:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:22:04.133218 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be74c33-e0a3-466a-8cae-d3d3c9072e0d None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:04.142982 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:04.222312 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:04.234360 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e5630d-20e0-4b72-9f72-a76780c37882 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:04.261357 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be74c33-e0a3-466a-8cae-d3d3c9072e0d None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:04.279496 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd14e29e-caf3-4eb9-99b5-a678bc0e626c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:04.280421 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd14e29e-caf3-4eb9-99b5-a678bc0e626c None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:04.280702 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd14e29e-caf3-4eb9-99b5-a678bc0e626c None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:04.280978 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd14e29e-caf3-4eb9-99b5-a678bc0e626c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:04.290562 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd14e29e-caf3-4eb9-99b5-a678bc0e626c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:04.292647 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd14e29e-caf3-4eb9-99b5-a678bc0e626c None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:04.300461 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7344b7fd-00fa-4c50-a193-e2c22d18ff4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:04.301075 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7344b7fd-00fa-4c50-a193-e2c22d18ff4e None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:04.301400 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7344b7fd-00fa-4c50-a193-e2c22d18ff4e None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:04.301672 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7344b7fd-00fa-4c50-a193-e2c22d18ff4e None admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:04.302320 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7344b7fd-00fa-4c50-a193-e2c22d18ff4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:04.302617 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7344b7fd-00fa-4c50-a193-e2c22d18ff4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:04.304169 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7344b7fd-00fa-4c50-a193-e2c22d18ff4e None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:04.306163 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-bd14e29e-caf3-4eb9-99b5-a678bc0e626c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 06:22:04.306163 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:04.306163 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:04.306163 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:04.306163 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:04.306163 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:04.306163 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:04.306163 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:04.306163 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:04.306163 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:04.306163 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:04.306163 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:04.306163 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 06:22:04.306163 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 06:22:04.306163 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 06:22:04.306163 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 06:22:04.307078 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:04.307078 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:04.307078 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 06:22:04.307078 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:22:04.307078 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 06:22:04.307078 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:22:04.307078 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:04.307078 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:04.307078 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:04.307078 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:04.307078 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:04.307078 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:04.307078 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:04.307078 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:04.307078 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:04.307078 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:04.307078 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:04.307078 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:04.308039 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:04.308039 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:04.308039 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 06:22:04.308039 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 06:22:04.308039 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 06:22:04.308039 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 06:22:04.308039 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 06:22:04.308039 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 06:22:04.308039 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 06:22:04.308039 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:04.313429 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 280/564] 10.222.0.47 () {56 vars in 1093 bytes} [Wed Aug 24 06:22:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:04.314380 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd14e29e-caf3-4eb9-99b5-a678bc0e626c None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:04.323944 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 285/565] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 24 06:22:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:04.325916 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7344b7fd-00fa-4c50-a193-e2c22d18ff4e None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:04.435139 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:04.441662 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd14e29e-caf3-4eb9-99b5-a678bc0e626c None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:04.452205 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7344b7fd-00fa-4c50-a193-e2c22d18ff4e None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:04.478833 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2198ef5f-6af1-4486-abca-bb99c228c826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:04.479543 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2198ef5f-6af1-4486-abca-bb99c228c826 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:04.479840 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2198ef5f-6af1-4486-abca-bb99c228c826 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:04.480116 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2198ef5f-6af1-4486-abca-bb99c228c826 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:04.488768 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:04.491181 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2198ef5f-6af1-4486-abca-bb99c228c826 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:04.492647 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2198ef5f-6af1-4486-abca-bb99c228c826 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:04.494703 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c70e929-e51b-42c2-a2d9-0139c812831e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:04.495834 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c70e929-e51b-42c2-a2d9-0139c812831e None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:04.496154 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c70e929-e51b-42c2-a2d9-0139c812831e None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:04.496521 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c70e929-e51b-42c2-a2d9-0139c812831e None admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:04.497421 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c70e929-e51b-42c2-a2d9-0139c812831e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:04.497671 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c70e929-e51b-42c2-a2d9-0139c812831e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:04.499122 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c70e929-e51b-42c2-a2d9-0139c812831e None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:04.511641 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-2198ef5f-6af1-4486-abca-bb99c228c826 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:04.511641 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:04.511641 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:04.511641 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:04.511641 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:04.511641 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:04.511641 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:04.511641 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:04.511641 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:04.511641 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:04.511641 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:04.511641 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:04.511641 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:22:04.511641 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:22:04.511641 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:22:04.511641 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:22:04.513021 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:04.513021 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:04.513021 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:04.513021 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:04.513021 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:04.513021 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:04.513021 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:04.513021 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:04.513021 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:04.513021 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:04.513021 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:04.513021 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:04.513021 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:04.513021 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:04.513021 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:04.513021 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:04.513021 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:22:04.514469 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:22:04.514469 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:04.514469 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:04.517500 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 281/566] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:22:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:04.518434 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2198ef5f-6af1-4486-abca-bb99c228c826 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:04.525295 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 286/567] 10.222.0.47 () {56 vars in 1133 bytes} [Wed Aug 24 06:22:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:04.525295 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c70e929-e51b-42c2-a2d9-0139c812831e None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:04.635504 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c70e929-e51b-42c2-a2d9-0139c812831e None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:04.650215 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2198ef5f-6af1-4486-abca-bb99c228c826 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:04.677515 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea7fcee-442f-4009-8b82-74ff53c2dffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:04.678254 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea7fcee-442f-4009-8b82-74ff53c2dffb None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:04.678545 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea7fcee-442f-4009-8b82-74ff53c2dffb None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:04.678870 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea7fcee-442f-4009-8b82-74ff53c2dffb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:04.688660 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688af79a-2350-4289-9945-a6349923cbd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:04.689565 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688af79a-2350-4289-9945-a6349923cbd9 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:04.689978 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688af79a-2350-4289-9945-a6349923cbd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:04.690189 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea7fcee-442f-4009-8b82-74ff53c2dffb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:22:04.690290 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688af79a-2350-4289-9945-a6349923cbd9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:04.694953 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-8ea7fcee-442f-4009-8b82-74ff53c2dffb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:22:04.694953 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:04.694953 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:04.694953 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:04.694953 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:04.694953 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:04.694953 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:04.694953 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:04.694953 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:04.694953 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:04.694953 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:04.694953 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:04.694953 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:22:04.694953 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:22:04.694953 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:22:04.694953 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:22:04.695904 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:22:04.695904 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:22:04.695904 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:22:04.695904 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:22:04.695904 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:04.695904 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:04.695904 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:04.695904 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:04.695904 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:04.695904 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:04.695904 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:04.695904 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:04.695904 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:04.695904 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:04.695904 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:04.695904 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:04.702147 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:04.702147 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:04.702147 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:04.702147 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:04.702147 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:22:04.702147 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:22:04.702147 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:22:04.702147 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:22:04.702147 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:22:04.702147 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:22:04.702147 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:22:04.702147 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:04.702147 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 287/568] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Aug 24 06:22:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:04.702147 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea7fcee-442f-4009-8b82-74ff53c2dffb None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:04.705154 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688af79a-2350-4289-9945-a6349923cbd9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:04.706682 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688af79a-2350-4289-9945-a6349923cbd9 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:04.722013 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-688af79a-2350-4289-9945-a6349923cbd9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:22:04.722013 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:04.722013 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:04.722013 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:04.722013 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:04.722013 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:04.722013 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:04.722013 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:04.722013 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:04.722013 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:04.722013 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:04.722013 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:04.722013 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 06:22:04.722013 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 06:22:04.722013 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 06:22:04.723111 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 06:22:04.723111 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:04.723111 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:04.723111 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:04.723111 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:04.723111 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:04.723111 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:04.723111 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:04.723111 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:04.723111 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:04.723111 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:04.723111 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:04.723111 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:04.723111 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:04.723111 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:04.723111 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:04.723111 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:04.723999 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 06:22:04.723999 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 06:22:04.723999 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:22:04.723999 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:22:04.723999 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:22:04.723999 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:22:04.723999 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:22:04.723999 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:04.723999 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 282/569] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 24 06:22:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:04.723999 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688af79a-2350-4289-9945-a6349923cbd9 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:04.737238 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:04.760454 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:04.816465 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea7fcee-442f-4009-8b82-74ff53c2dffb None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:04.831013 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688af79a-2350-4289-9945-a6349923cbd9 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:04.854100 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d10935a-d648-4812-9df8-e723afeb9b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:04.855284 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d10935a-d648-4812-9df8-e723afeb9b76 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:04.855572 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d10935a-d648-4812-9df8-e723afeb9b76 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:04.855850 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d10935a-d648-4812-9df8-e723afeb9b76 None admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:04.856494 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d10935a-d648-4812-9df8-e723afeb9b76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:04.856804 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d10935a-d648-4812-9df8-e723afeb9b76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:04.858227 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d10935a-d648-4812-9df8-e723afeb9b76 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:04.867949 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d387ad2-4e21-48fe-a6a8-f42f8dbe95cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:04.868578 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d387ad2-4e21-48fe-a6a8-f42f8dbe95cf None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:04.868856 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d387ad2-4e21-48fe-a6a8-f42f8dbe95cf None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:04.869127 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d387ad2-4e21-48fe-a6a8-f42f8dbe95cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:04.869716 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 288/570] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 24 06:22:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:04.870674 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d10935a-d648-4812-9df8-e723afeb9b76 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:04.880821 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d387ad2-4e21-48fe-a6a8-f42f8dbe95cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:04.882292 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d387ad2-4e21-48fe-a6a8-f42f8dbe95cf None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:04.896689 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-2d387ad2-4e21-48fe-a6a8-f42f8dbe95cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:04.896689 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:04.896689 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:04.896689 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:04.896689 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:04.896689 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:04.896689 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:04.896689 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:04.896689 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:04.896689 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:04.896689 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:04.896689 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:04.896689 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:22:04.896689 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:22:04.896689 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:22:04.896689 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:22:04.897811 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:04.897811 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:04.897811 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:04.897811 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:04.897811 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:04.897811 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:04.897811 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:04.897811 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:04.897811 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:04.897811 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:04.897811 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:04.897811 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:04.897811 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:04.897811 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:04.897811 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:04.897811 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:04.897811 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:22:04.898697 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:22:04.898697 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:04.898697 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:04.898697 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 283/571] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:22:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:04.898697 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d387ad2-4e21-48fe-a6a8-f42f8dbe95cf None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:04.970783 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:04.986868 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d10935a-d648-4812-9df8-e723afeb9b76 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:04.997847 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:05.015505 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d387ad2-4e21-48fe-a6a8-f42f8dbe95cf None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:05.021449 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a95eefc-aa9e-4d72-9eb1-d0ac71c931ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:05.022124 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a95eefc-aa9e-4d72-9eb1-d0ac71c931ca None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:05.022410 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a95eefc-aa9e-4d72-9eb1-d0ac71c931ca None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:05.022702 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a95eefc-aa9e-4d72-9eb1-d0ac71c931ca None admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:05.023364 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a95eefc-aa9e-4d72-9eb1-d0ac71c931ca None admin] RBAC: Adding query filter params (name=service) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:05.023667 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a95eefc-aa9e-4d72-9eb1-d0ac71c931ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:05.025398 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a95eefc-aa9e-4d72-9eb1-d0ac71c931ca None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:05.035161 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 289/572] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Aug 24 06:22:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:05.036428 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce5917e-4911-4d62-ac75-deb900430e00 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:05.036719 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce5917e-4911-4d62-ac75-deb900430e00 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:05.037292 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce5917e-4911-4d62-ac75-deb900430e00 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:05.039036 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 290/573] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:22:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:22:05.039941 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce5917e-4911-4d62-ac75-deb900430e00 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:05.054369 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4668bac7-2201-4e05-8ae2-b91056914d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:05.054993 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4668bac7-2201-4e05-8ae2-b91056914d40 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:05.055284 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4668bac7-2201-4e05-8ae2-b91056914d40 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:05.055540 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4668bac7-2201-4e05-8ae2-b91056914d40 None admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:05.056231 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4668bac7-2201-4e05-8ae2-b91056914d40 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:05.056534 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4668bac7-2201-4e05-8ae2-b91056914d40 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:05.058316 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4668bac7-2201-4e05-8ae2-b91056914d40 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:05.070196 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 284/574] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Aug 24 06:22:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:05.071023 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4668bac7-2201-4e05-8ae2-b91056914d40 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:05.149688 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce5917e-4911-4d62-ac75-deb900430e00 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:05.175101 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4668bac7-2201-4e05-8ae2-b91056914d40 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:05.184268 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af27fdc-2cff-41e7-8727-10d13938054d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:05.184909 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af27fdc-2cff-41e7-8727-10d13938054d None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:05.185212 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af27fdc-2cff-41e7-8727-10d13938054d None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:05.185487 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af27fdc-2cff-41e7-8727-10d13938054d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:05.193733 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af27fdc-2cff-41e7-8727-10d13938054d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:05.195196 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af27fdc-2cff-41e7-8727-10d13938054d None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:05.195324 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:05.209881 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-3af27fdc-2cff-41e7-8727-10d13938054d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 06:22:05.209881 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:05.209881 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:05.209881 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:05.209881 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:05.209881 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:05.209881 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:05.209881 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:05.209881 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:05.209881 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:05.209881 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:05.209881 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:05.209881 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 06:22:05.209881 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 06:22:05.209881 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 06:22:05.209881 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 06:22:05.212142 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:05.212142 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:05.212142 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 06:22:05.212142 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:22:05.212142 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 06:22:05.212142 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:22:05.212142 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:05.212142 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:05.212142 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:05.212142 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:05.212142 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:05.212142 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:05.212142 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:05.212142 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:05.212142 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:05.212142 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:05.212142 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:05.212142 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:05.213102 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:05.213102 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:05.213102 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 06:22:05.213102 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 06:22:05.213102 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 06:22:05.213102 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 06:22:05.213102 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 06:22:05.213102 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 06:22:05.213102 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 06:22:05.213102 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:05.213795 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd79f29a-be43-4d35-b057-043d6874ea8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:05.213795 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd79f29a-be43-4d35-b057-043d6874ea8a None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:05.213795 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd79f29a-be43-4d35-b057-043d6874ea8a None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:05.214590 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 291/575] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 24 06:22:05 2022] GET /identity/v3/users/cinder => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:05.214736 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd79f29a-be43-4d35-b057-043d6874ea8a None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:05.214868 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af27fdc-2cff-41e7-8727-10d13938054d None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:05.215002 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd79f29a-be43-4d35-b057-043d6874ea8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:05.215002 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd79f29a-be43-4d35-b057-043d6874ea8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:05.215002 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd79f29a-be43-4d35-b057-043d6874ea8a None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:05.226446 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 285/576] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 24 06:22:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:05.228255 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3a3e9a-a5db-4570-b937-dbbf196bc4e9 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:05.228769 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3a3e9a-a5db-4570-b937-dbbf196bc4e9 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:05.229299 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3a3e9a-a5db-4570-b937-dbbf196bc4e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:05.248721 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:05.249153 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-7d3a3e9a-a5db-4570-b937-dbbf196bc4e9 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:22:05.250422 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-7d3a3e9a-a5db-4570-b937-dbbf196bc4e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:22:05.320098 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 286/577] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:22:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:22:05.321111 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3a3e9a-a5db-4570-b937-dbbf196bc4e9 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:05.330139 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af27fdc-2cff-41e7-8727-10d13938054d None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:05.377484 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355e80df-1218-4a75-a464-c1264bcfcfc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:05.378123 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355e80df-1218-4a75-a464-c1264bcfcfc1 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:05.378452 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355e80df-1218-4a75-a464-c1264bcfcfc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:05.378736 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355e80df-1218-4a75-a464-c1264bcfcfc1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:05.391436 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355e80df-1218-4a75-a464-c1264bcfcfc1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:05.392901 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355e80df-1218-4a75-a464-c1264bcfcfc1 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:05.414361 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-355e80df-1218-4a75-a464-c1264bcfcfc1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:05.414361 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:05.414361 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:05.414361 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:05.414361 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:05.414361 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:05.414361 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:05.414361 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:05.414361 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:05.414361 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:05.414361 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:05.414361 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:05.414361 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:22:05.414361 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:22:05.414361 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:22:05.414361 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:22:05.415369 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:05.415369 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:05.415369 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:05.415369 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:05.415369 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:05.415369 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:05.415369 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:05.415369 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:05.415369 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:05.415369 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:05.415369 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:05.415369 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:05.415369 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:05.415369 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:05.415369 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:05.415369 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:05.415369 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:22:05.416278 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:22:05.416278 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:05.416278 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:05.417969 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 292/578] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:22:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:05.419010 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355e80df-1218-4a75-a464-c1264bcfcfc1 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:05.452005 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3a3e9a-a5db-4570-b937-dbbf196bc4e9 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:05.467121 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:05.494624 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26571b60-8dd8-45e5-a9d8-a58933205d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:05.495814 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26571b60-8dd8-45e5-a9d8-a58933205d1a None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:05.496099 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26571b60-8dd8-45e5-a9d8-a58933205d1a None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:05.496407 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26571b60-8dd8-45e5-a9d8-a58933205d1a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:05.497102 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26571b60-8dd8-45e5-a9d8-a58933205d1a None admin] RBAC: Adding query filter params (role.id=1712d85aedaf4189bf4bc89094d888c3, scope.project.id=c02e49a89cdd40d885bfa816f06c58a1, user.id=02f1cbaf2cd6427585f298c9d58c319a) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:05.497440 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26571b60-8dd8-45e5-a9d8-a58933205d1a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:05.498950 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26571b60-8dd8-45e5-a9d8-a58933205d1a None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:05.505554 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 287/579] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 24 06:22:05 2022] GET /identity/v3/role_assignments?user.id=02f1cbaf2cd6427585f298c9d58c319a&scope.project.id=c02e49a89cdd40d885bfa816f06c58a1&role.id=1712d85aedaf4189bf4bc89094d888c3 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:05.507165 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26571b60-8dd8-45e5-a9d8-a58933205d1a None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:05.555580 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:05.575941 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355e80df-1218-4a75-a464-c1264bcfcfc1 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:05.619592 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8619731-6ac8-4c24-ac1c-f31245957300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:05.621002 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8619731-6ac8-4c24-ac1c-f31245957300 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:05.621543 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8619731-6ac8-4c24-ac1c-f31245957300 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:05.622023 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8619731-6ac8-4c24-ac1c-f31245957300 None admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:05.623391 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8619731-6ac8-4c24-ac1c-f31245957300 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:05.623946 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8619731-6ac8-4c24-ac1c-f31245957300 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:05.626803 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8619731-6ac8-4c24-ac1c-f31245957300 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:05.640486 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26571b60-8dd8-45e5-a9d8-a58933205d1a None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:05.653256 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 293/580] 10.222.0.47 () {56 vars in 1137 bytes} [Wed Aug 24 06:22:05 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:05.653256 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8619731-6ac8-4c24-ac1c-f31245957300 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:05.677342 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b650ae-52fe-4f09-b46e-b782d9579b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:05.678149 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b650ae-52fe-4f09-b46e-b782d9579b4b None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:05.678600 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b650ae-52fe-4f09-b46e-b782d9579b4b None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:05.678971 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b650ae-52fe-4f09-b46e-b782d9579b4b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:05.688313 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b650ae-52fe-4f09-b46e-b782d9579b4b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:05.690767 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b650ae-52fe-4f09-b46e-b782d9579b4b None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:05.703347 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-b8b650ae-52fe-4f09-b46e-b782d9579b4b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:22:05.703347 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:05.703347 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:05.703347 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:05.703347 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:05.703347 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:05.703347 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:05.703347 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:05.703347 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:05.703347 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:05.703347 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:05.703347 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:05.703347 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 06:22:05.703347 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 06:22:05.703347 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 06:22:05.704445 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 06:22:05.704445 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:05.704445 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:05.704445 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:05.704445 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:05.704445 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:05.704445 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:05.704445 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:05.704445 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:05.704445 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:05.704445 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:05.704445 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:05.704445 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:05.704445 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:05.704445 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:05.704445 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:05.704445 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:05.705523 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 06:22:05.705523 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 06:22:05.705523 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:22:05.705523 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:22:05.705523 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:22:05.705523 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:22:05.705523 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:22:05.705523 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:05.705523 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 288/581] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 24 06:22:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:05.705523 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b650ae-52fe-4f09-b46e-b782d9579b4b None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:05.743879 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:05.777979 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8619731-6ac8-4c24-ac1c-f31245957300 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:05.822989 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdfce51-42c8-4968-b2d5-a8a6f5758d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:05.823620 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdfce51-42c8-4968-b2d5-a8a6f5758d19 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:05.823902 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdfce51-42c8-4968-b2d5-a8a6f5758d19 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:05.824288 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdfce51-42c8-4968-b2d5-a8a6f5758d19 None admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:05.824543 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b650ae-52fe-4f09-b46e-b782d9579b4b None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:05.824939 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdfce51-42c8-4968-b2d5-a8a6f5758d19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:05.825264 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdfce51-42c8-4968-b2d5-a8a6f5758d19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:05.826640 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdfce51-42c8-4968-b2d5-a8a6f5758d19 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:05.844171 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 294/582] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 24 06:22:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:05.844171 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdfce51-42c8-4968-b2d5-a8a6f5758d19 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:05.844171 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:05.872469 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7635b32e-50c8-42c5-8d6f-3b132400eec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:05.873247 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7635b32e-50c8-42c5-8d6f-3b132400eec2 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:05.873598 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7635b32e-50c8-42c5-8d6f-3b132400eec2 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:05.873935 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7635b32e-50c8-42c5-8d6f-3b132400eec2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:05.886211 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7635b32e-50c8-42c5-8d6f-3b132400eec2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:22:05.891927 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-7635b32e-50c8-42c5-8d6f-3b132400eec2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:22:05.891927 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:05.891927 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:05.891927 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:05.891927 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:05.891927 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:05.891927 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:05.891927 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:05.891927 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:05.891927 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:05.891927 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:05.891927 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:05.891927 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:22:05.891927 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:22:05.891927 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:22:05.891927 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:22:05.892873 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:22:05.892873 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:22:05.892873 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:22:05.892873 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:22:05.892873 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:05.892873 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:05.892873 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:05.892873 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:05.892873 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:05.892873 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:05.892873 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:05.892873 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:05.892873 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:05.892873 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:05.892873 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:05.892873 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:05.893756 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:05.893756 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:05.893756 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:05.893756 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:05.893756 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:22:05.893756 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:22:05.893756 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:22:05.893756 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:22:05.893756 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:22:05.893756 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:22:05.893756 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:22:05.893756 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:05.893756 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 289/583] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Aug 24 06:22:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:05.895298 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7635b32e-50c8-42c5-8d6f-3b132400eec2 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:05.967488 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdfce51-42c8-4968-b2d5-a8a6f5758d19 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:06.006390 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf8a478-21ee-4be8-9177-a1dfc3adf0bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:06.007028 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf8a478-21ee-4be8-9177-a1dfc3adf0bd None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:06.007354 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf8a478-21ee-4be8-9177-a1dfc3adf0bd None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:06.007558 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf8a478-21ee-4be8-9177-a1dfc3adf0bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:06.008583 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:06.014767 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7635b32e-50c8-42c5-8d6f-3b132400eec2 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:06.018754 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf8a478-21ee-4be8-9177-a1dfc3adf0bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:06.025794 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf8a478-21ee-4be8-9177-a1dfc3adf0bd None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:06.035354 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-fbf8a478-21ee-4be8-9177-a1dfc3adf0bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:06.035354 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:06.035354 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:06.035354 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:06.035354 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:06.035354 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:06.035354 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:06.035354 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:06.035354 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:06.035354 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:06.035354 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:06.035354 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:06.035354 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:22:06.035354 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:22:06.035354 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:22:06.035354 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:22:06.036489 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:06.036489 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:06.036489 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:06.036489 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:06.036489 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:06.036489 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:06.036489 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:06.036489 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:06.036489 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:06.036489 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:06.036489 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:06.036489 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:06.036489 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:06.036489 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:06.036489 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:06.036489 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:06.036489 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:22:06.037650 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:22:06.037650 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:06.037650 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:06.037650 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 295/584] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:22:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:06.037650 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf8a478-21ee-4be8-9177-a1dfc3adf0bd None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:06.060012 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95217aa0-417f-49d8-8519-6013d51c47ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:06.060628 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95217aa0-417f-49d8-8519-6013d51c47ac None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:06.060907 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95217aa0-417f-49d8-8519-6013d51c47ac None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:06.061208 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95217aa0-417f-49d8-8519-6013d51c47ac None admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:06.061900 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95217aa0-417f-49d8-8519-6013d51c47ac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:06.062203 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95217aa0-417f-49d8-8519-6013d51c47ac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:06.063636 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95217aa0-417f-49d8-8519-6013d51c47ac None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:06.075013 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 290/585] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Aug 24 06:22:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:06.075840 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95217aa0-417f-49d8-8519-6013d51c47ac None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:06.100377 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:06.150649 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf8a478-21ee-4be8-9177-a1dfc3adf0bd None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:06.185855 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95217aa0-417f-49d8-8519-6013d51c47ac None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:06.190605 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6c8caf-22b9-403b-a5d3-1a3d34909e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:06.191243 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6c8caf-22b9-403b-a5d3-1a3d34909e34 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:06.191521 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6c8caf-22b9-403b-a5d3-1a3d34909e34 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:06.191792 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6c8caf-22b9-403b-a5d3-1a3d34909e34 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:06.201906 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6c8caf-22b9-403b-a5d3-1a3d34909e34 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:06.203405 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6c8caf-22b9-403b-a5d3-1a3d34909e34 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:06.215755 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-fa6c8caf-22b9-403b-a5d3-1a3d34909e34 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 06:22:06.215755 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:06.215755 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:06.215755 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:06.215755 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:06.215755 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:06.215755 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:06.215755 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:06.215755 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:06.215755 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:06.215755 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:06.215755 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:06.215755 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 06:22:06.215755 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 06:22:06.215755 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 06:22:06.215755 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 06:22:06.216919 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:06.216919 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:06.216919 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 06:22:06.216919 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:22:06.216919 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 06:22:06.216919 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:22:06.216919 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:06.216919 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:06.216919 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:06.216919 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:06.216919 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:06.216919 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:06.216919 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:06.216919 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:06.216919 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:06.216919 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:06.216919 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:06.216919 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:06.218118 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:06.218118 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:06.218118 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 06:22:06.218118 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 06:22:06.218118 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 06:22:06.218118 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 06:22:06.218118 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 06:22:06.218118 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 06:22:06.218118 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 06:22:06.218118 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:06.218118 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 296/586] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 24 06:22:06 2022] GET /identity/v3/users/glance => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:06.218118 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6c8caf-22b9-403b-a5d3-1a3d34909e34 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:06.228501 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f908d9-4543-4b01-ab50-8ce4ae5b8a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:06.229861 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f908d9-4543-4b01-ab50-8ce4ae5b8a50 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:06.230244 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f908d9-4543-4b01-ab50-8ce4ae5b8a50 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:06.230723 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f908d9-4543-4b01-ab50-8ce4ae5b8a50 None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:06.231402 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f908d9-4543-4b01-ab50-8ce4ae5b8a50 None admin] RBAC: Adding query filter params (name=service) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:06.231709 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f908d9-4543-4b01-ab50-8ce4ae5b8a50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:06.233120 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f908d9-4543-4b01-ab50-8ce4ae5b8a50 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:06.241517 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 291/587] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Aug 24 06:22:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:06.242176 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f908d9-4543-4b01-ab50-8ce4ae5b8a50 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:06.262457 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:06.338402 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6c8caf-22b9-403b-a5d3-1a3d34909e34 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:06.354190 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f908d9-4543-4b01-ab50-8ce4ae5b8a50 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:06.356070 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:06.379750 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b87df3-6dca-494f-8360-83858f70906e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:06.380375 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b87df3-6dca-494f-8360-83858f70906e None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:06.380912 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b87df3-6dca-494f-8360-83858f70906e None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:06.381211 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b87df3-6dca-494f-8360-83858f70906e None admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:06.381908 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b87df3-6dca-494f-8360-83858f70906e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:06.382348 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b87df3-6dca-494f-8360-83858f70906e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:06.384319 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b87df3-6dca-494f-8360-83858f70906e None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:06.391428 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9fec25-bf6e-46c4-a06e-d024b1e4efb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:06.392801 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9fec25-bf6e-46c4-a06e-d024b1e4efb9 None admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:06.393078 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9fec25-bf6e-46c4-a06e-d024b1e4efb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:06.393392 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9fec25-bf6e-46c4-a06e-d024b1e4efb9 None admin] PATH_INFO: `/v3/projects/c02e49a89cdd40d885bfa816f06c58a1/users/1f4f49c1b08c460f8ba6c451c9c92c2a/roles/1712d85aedaf4189bf4bc89094d888c3` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:06.397033 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 297/588] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 24 06:22:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:06.397893 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b87df3-6dca-494f-8360-83858f70906e None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:06.414512 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9fec25-bf6e-46c4-a06e-d024b1e4efb9 None admin] RBAC: Authorizing `identity:create_grant(project_id=c02e49a89cdd40d885bfa816f06c58a1, user_id=1f4f49c1b08c460f8ba6c451c9c92c2a, role_id=1712d85aedaf4189bf4bc89094d888c3)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:06.415948 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9fec25-bf6e-46c4-a06e-d024b1e4efb9 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:06.502036 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b87df3-6dca-494f-8360-83858f70906e None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:06.537898 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cb97fd-16ce-43df-b0e0-1212016b282e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:06.538500 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cb97fd-16ce-43df-b0e0-1212016b282e None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:06.538776 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cb97fd-16ce-43df-b0e0-1212016b282e None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:06.539051 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cb97fd-16ce-43df-b0e0-1212016b282e None admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:06.539808 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cb97fd-16ce-43df-b0e0-1212016b282e None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:06.540105 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cb97fd-16ce-43df-b0e0-1212016b282e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:06.541559 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cb97fd-16ce-43df-b0e0-1212016b282e None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:06.561435 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 292/589] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Aug 24 06:22:06 2022] PUT /identity/v3/projects/c02e49a89cdd40d885bfa816f06c58a1/users/1f4f49c1b08c460f8ba6c451c9c92c2a/roles/1712d85aedaf4189bf4bc89094d888c3 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:22:06.561435 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9fec25-bf6e-46c4-a06e-d024b1e4efb9 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:06.574368 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 298/590] 10.222.0.47 () {56 vars in 1137 bytes} [Wed Aug 24 06:22:06 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:06.575512 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cb97fd-16ce-43df-b0e0-1212016b282e None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:06.630353 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:06.667755 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:06.723065 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cb97fd-16ce-43df-b0e0-1212016b282e None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:06.723268 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9fec25-bf6e-46c4-a06e-d024b1e4efb9 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:06.769504 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb9cfeb-5b54-41bb-9ab0-1a75ea956519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:06.770100 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb9cfeb-5b54-41bb-9ab0-1a75ea956519 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:06.770375 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb9cfeb-5b54-41bb-9ab0-1a75ea956519 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:06.770646 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb9cfeb-5b54-41bb-9ab0-1a75ea956519 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:06.771805 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5f022a-569d-417b-ad93-dd611f16e242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:06.772441 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5f022a-569d-417b-ad93-dd611f16e242 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:06.772716 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5f022a-569d-417b-ad93-dd611f16e242 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:06.772979 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5f022a-569d-417b-ad93-dd611f16e242 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:06.779989 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb9cfeb-5b54-41bb-9ab0-1a75ea956519 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:06.781998 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb9cfeb-5b54-41bb-9ab0-1a75ea956519 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:06.796386 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5f022a-569d-417b-ad93-dd611f16e242 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:06.796386 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5f022a-569d-417b-ad93-dd611f16e242 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:06.796755 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-acb9cfeb-5b54-41bb-9ab0-1a75ea956519 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:22:06.796755 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:06.796755 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:06.796755 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:06.796755 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:06.796755 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:06.796755 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:06.796755 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:06.796755 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:06.796755 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:06.796755 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:06.796755 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:06.796755 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 06:22:06.796755 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 06:22:06.796755 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 06:22:06.797639 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 06:22:06.797639 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:06.797639 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:06.797639 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:06.797639 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:06.797639 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:06.797639 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:06.797639 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:06.797639 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:06.797639 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:06.797639 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:06.797639 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:06.797639 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:06.797639 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:06.797639 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:06.797639 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:06.797639 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:06.798528 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 06:22:06.798528 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 06:22:06.798528 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:22:06.798528 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:22:06.798528 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:22:06.798528 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:22:06.798528 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:22:06.798528 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:06.798528 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 299/591] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 24 06:22:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:06.799448 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb9cfeb-5b54-41bb-9ab0-1a75ea956519 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:06.801526 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-0f5f022a-569d-417b-ad93-dd611f16e242 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:06.801526 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:06.801526 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:06.801526 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:06.801526 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:06.801526 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:06.801526 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:06.801526 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:06.801526 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:06.801526 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:06.801526 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:06.801526 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:06.801526 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:22:06.801526 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:22:06.801526 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:22:06.801526 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:22:06.802457 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:06.802457 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:06.802457 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:06.802457 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:06.802457 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:06.802457 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:06.802457 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:06.802457 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:06.802457 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:06.802457 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:06.802457 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:06.802457 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:06.802457 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:06.802457 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:06.802457 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:06.802457 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:06.802457 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:22:06.803964 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:22:06.803964 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:06.803964 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:06.803964 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 293/592] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:22:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:06.814048 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5f022a-569d-417b-ad93-dd611f16e242 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:06.918504 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:06.929786 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb9cfeb-5b54-41bb-9ab0-1a75ea956519 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:06.959240 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5f022a-569d-417b-ad93-dd611f16e242 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:06.978920 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:06.989315 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4720705-573c-4ef5-a787-c07e1f206907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:06.990452 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4720705-573c-4ef5-a787-c07e1f206907 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:06.996760 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4720705-573c-4ef5-a787-c07e1f206907 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:06.996760 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4720705-573c-4ef5-a787-c07e1f206907 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:06.998683 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e15d94f-1ec3-4b84-ae66-3307b6042edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:06.999327 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e15d94f-1ec3-4b84-ae66-3307b6042edb None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:06.999618 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e15d94f-1ec3-4b84-ae66-3307b6042edb None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:06.999894 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e15d94f-1ec3-4b84-ae66-3307b6042edb None admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:07.000623 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e15d94f-1ec3-4b84-ae66-3307b6042edb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:07.000931 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e15d94f-1ec3-4b84-ae66-3307b6042edb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:07.002889 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e15d94f-1ec3-4b84-ae66-3307b6042edb None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:07.007467 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4720705-573c-4ef5-a787-c07e1f206907 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:07.009685 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4720705-573c-4ef5-a787-c07e1f206907 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:07.022031 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 294/593] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Aug 24 06:22:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:07.022031 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e15d94f-1ec3-4b84-ae66-3307b6042edb None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:07.036184 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-a4720705-573c-4ef5-a787-c07e1f206907 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:07.036184 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:07.036184 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:07.036184 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:07.036184 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:07.036184 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:07.036184 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:07.036184 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:07.036184 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:07.036184 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:07.036184 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:07.036184 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:07.036184 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:22:07.036184 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:22:07.036184 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:22:07.036184 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:22:07.037208 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:07.037208 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:07.037208 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:07.037208 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:07.037208 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:07.037208 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:07.037208 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:07.037208 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:07.037208 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:07.037208 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:07.037208 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:07.037208 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:07.037208 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:07.037208 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:07.037208 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:07.037208 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:07.037208 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:22:07.038140 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:22:07.038140 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:07.038140 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:07.039018 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 300/594] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:22:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:07.039248 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4720705-573c-4ef5-a787-c07e1f206907 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:07.137950 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e15d94f-1ec3-4b84-ae66-3307b6042edb None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:07.153606 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4720705-573c-4ef5-a787-c07e1f206907 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:07.176995 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef90247-b8b2-47b7-8970-c44299b78c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:07.178082 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef90247-b8b2-47b7-8970-c44299b78c3f None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:07.178688 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef90247-b8b2-47b7-8970-c44299b78c3f None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:07.179198 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef90247-b8b2-47b7-8970-c44299b78c3f None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:07.180061 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef90247-b8b2-47b7-8970-c44299b78c3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:07.180572 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef90247-b8b2-47b7-8970-c44299b78c3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:07.182565 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef90247-b8b2-47b7-8970-c44299b78c3f None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:07.190124 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7fa0e5-dcbc-40a1-a9ec-bde9f0ae7771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:07.190802 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7fa0e5-dcbc-40a1-a9ec-bde9f0ae7771 None admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:07.191085 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7fa0e5-dcbc-40a1-a9ec-bde9f0ae7771 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:07.191363 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7fa0e5-dcbc-40a1-a9ec-bde9f0ae7771 None admin] PATH_INFO: `/v3/projects/c02e49a89cdd40d885bfa816f06c58a1/users/36639dce4f97412f89d71367f650b901/roles/1712d85aedaf4189bf4bc89094d888c3` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:07.198314 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 295/595] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 24 06:22:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:07.199616 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef90247-b8b2-47b7-8970-c44299b78c3f None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:07.207910 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7fa0e5-dcbc-40a1-a9ec-bde9f0ae7771 None admin] RBAC: Authorizing `identity:create_grant(project_id=c02e49a89cdd40d885bfa816f06c58a1, user_id=36639dce4f97412f89d71367f650b901, role_id=1712d85aedaf4189bf4bc89094d888c3)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:07.209409 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7fa0e5-dcbc-40a1-a9ec-bde9f0ae7771 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:07.211697 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:07.246645 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:07.314272 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef90247-b8b2-47b7-8970-c44299b78c3f None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:07.350205 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4860fb30-9830-434b-b281-77d78968c070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:07.350960 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4860fb30-9830-434b-b281-77d78968c070 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:07.351247 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4860fb30-9830-434b-b281-77d78968c070 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:07.351519 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4860fb30-9830-434b-b281-77d78968c070 None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:07.352227 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4860fb30-9830-434b-b281-77d78968c070 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:07.352526 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4860fb30-9830-434b-b281-77d78968c070 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:07.353960 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4860fb30-9830-434b-b281-77d78968c070 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:07.367692 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 301/596] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Aug 24 06:22:07 2022] PUT /identity/v3/projects/c02e49a89cdd40d885bfa816f06c58a1/users/36639dce4f97412f89d71367f650b901/roles/1712d85aedaf4189bf4bc89094d888c3 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:22:07.367692 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7fa0e5-dcbc-40a1-a9ec-bde9f0ae7771 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:07.393238 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 296/597] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 24 06:22:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:07.400118 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4860fb30-9830-434b-b281-77d78968c070 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:07.552586 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7fa0e5-dcbc-40a1-a9ec-bde9f0ae7771 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:07.567625 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:07.571412 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4860fb30-9830-434b-b281-77d78968c070 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:07.606674 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce5b955-c95e-469b-84c9-4a2f992bcb93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:07.607360 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce5b955-c95e-469b-84c9-4a2f992bcb93 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:07.607611 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce5b955-c95e-469b-84c9-4a2f992bcb93 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:07.607883 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce5b955-c95e-469b-84c9-4a2f992bcb93 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:07.616523 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce5b955-c95e-469b-84c9-4a2f992bcb93 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:07.618032 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e671221-b79f-45f4-a126-051255787946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:07.618758 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce5b955-c95e-469b-84c9-4a2f992bcb93 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:07.618877 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e671221-b79f-45f4-a126-051255787946 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:07.619120 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e671221-b79f-45f4-a126-051255787946 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:07.619492 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e671221-b79f-45f4-a126-051255787946 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:07.627992 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e671221-b79f-45f4-a126-051255787946 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:07.629617 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e671221-b79f-45f4-a126-051255787946 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:07.634957 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-2ce5b955-c95e-469b-84c9-4a2f992bcb93 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 06:22:07.634957 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:07.634957 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:07.634957 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:07.634957 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:07.634957 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:07.634957 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:07.634957 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:07.634957 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:07.634957 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:07.634957 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:07.634957 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:07.634957 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 06:22:07.634957 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 06:22:07.634957 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 06:22:07.634957 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 06:22:07.635985 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:07.635985 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:07.635985 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 06:22:07.635985 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:22:07.635985 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 06:22:07.635985 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:22:07.635985 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:07.635985 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:07.635985 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:07.635985 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:07.635985 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:07.635985 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:07.635985 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:07.635985 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:07.635985 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:07.635985 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:07.635985 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:07.635985 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:07.636913 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:07.636913 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:07.636913 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 06:22:07.636913 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 06:22:07.636913 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 06:22:07.636913 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 06:22:07.636913 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 06:22:07.636913 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 06:22:07.636913 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 06:22:07.636913 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:07.639752 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 302/598] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Aug 24 06:22:07 2022] GET /identity/v3/users/neutron => generated 85 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:07.641321 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e505dd46-2dd7-49dc-97dc-4f8a2151e85e None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:07.641792 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e505dd46-2dd7-49dc-97dc-4f8a2151e85e None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:07.642229 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e505dd46-2dd7-49dc-97dc-4f8a2151e85e None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:07.643964 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 303/599] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:22:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:22:07.647582 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-0e671221-b79f-45f4-a126-051255787946 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:22:07.647582 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:07.647582 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:07.647582 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:07.647582 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:07.647582 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:07.647582 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:07.647582 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:07.647582 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:07.647582 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:07.647582 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:07.647582 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:07.647582 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 06:22:07.647582 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 06:22:07.647582 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 06:22:07.648464 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 06:22:07.648464 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:07.648464 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:07.648464 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:07.648464 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:07.648464 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:07.648464 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:07.648464 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:07.648464 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:07.648464 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:07.648464 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:07.648464 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:07.648464 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:07.648464 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:07.648464 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:07.648464 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:07.648464 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:07.649644 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 06:22:07.649644 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 06:22:07.649644 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:22:07.649644 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:22:07.649644 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:22:07.649644 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:22:07.649644 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:22:07.649644 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:07.650079 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 297/600] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 24 06:22:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:07.652371 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e671221-b79f-45f4-a126-051255787946 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:07.653095 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f20a5f0-2eaf-445e-9614-1524e4e8e296 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:07.653706 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f20a5f0-2eaf-445e-9614-1524e4e8e296 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:07.654158 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f20a5f0-2eaf-445e-9614-1524e4e8e296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:07.673863 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-9f20a5f0-2eaf-445e-9614-1524e4e8e296 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:22:07.675537 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-9f20a5f0-2eaf-445e-9614-1524e4e8e296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:22:07.750465 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:07.769093 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e671221-b79f-45f4-a126-051255787946 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:07.770328 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 304/601] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:22:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:22:07.771388 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f20a5f0-2eaf-445e-9614-1524e4e8e296 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:07.835487 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18e5267-681b-46fc-9229-46dd1f99789a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:07.837144 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18e5267-681b-46fc-9229-46dd1f99789a None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:07.837586 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18e5267-681b-46fc-9229-46dd1f99789a None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:07.838108 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18e5267-681b-46fc-9229-46dd1f99789a None admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:07.839126 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18e5267-681b-46fc-9229-46dd1f99789a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:07.839624 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18e5267-681b-46fc-9229-46dd1f99789a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:07.841311 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18e5267-681b-46fc-9229-46dd1f99789a None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:07.875154 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 298/602] 10.222.0.47 () {56 vars in 1139 bytes} [Wed Aug 24 06:22:07 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:07.880930 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18e5267-681b-46fc-9229-46dd1f99789a None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:07.912305 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:07.927012 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f20a5f0-2eaf-445e-9614-1524e4e8e296 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:08.002895 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda44c20-7b4d-40f9-a17e-a4bf4cb31d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:08.003513 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda44c20-7b4d-40f9-a17e-a4bf4cb31d3a None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:08.003790 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda44c20-7b4d-40f9-a17e-a4bf4cb31d3a None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:08.004054 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda44c20-7b4d-40f9-a17e-a4bf4cb31d3a None admin] PATH_INFO: `/v3/projects` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:08.004759 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda44c20-7b4d-40f9-a17e-a4bf4cb31d3a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:08.005063 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda44c20-7b4d-40f9-a17e-a4bf4cb31d3a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:08.006535 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda44c20-7b4d-40f9-a17e-a4bf4cb31d3a None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:08.042090 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 305/603] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Aug 24 06:22:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:08.044215 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda44c20-7b4d-40f9-a17e-a4bf4cb31d3a None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:08.071209 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18e5267-681b-46fc-9229-46dd1f99789a None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:08.107376 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a8e642-1436-40d7-a8b5-5d13265edfbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:08.108027 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a8e642-1436-40d7-a8b5-5d13265edfbb None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:08.108312 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a8e642-1436-40d7-a8b5-5d13265edfbb None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:08.108591 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a8e642-1436-40d7-a8b5-5d13265edfbb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:08.119355 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a8e642-1436-40d7-a8b5-5d13265edfbb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:08.121127 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a8e642-1436-40d7-a8b5-5d13265edfbb None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:08.123178 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:08.143783 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-06a8e642-1436-40d7-a8b5-5d13265edfbb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:08.143783 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:08.143783 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:08.143783 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:08.143783 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:08.143783 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:08.143783 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:08.143783 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:08.143783 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:08.143783 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:08.143783 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:08.143783 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:08.143783 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:22:08.143783 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:22:08.143783 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:22:08.143783 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:22:08.144779 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:08.144779 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:08.144779 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:08.144779 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:08.144779 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:08.144779 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:08.144779 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:08.144779 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:08.144779 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:08.144779 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:08.144779 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:08.144779 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:08.144779 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:08.144779 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:08.144779 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:08.144779 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:08.144779 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:22:08.145701 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:22:08.145701 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:08.145701 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:08.150259 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 299/604] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:22:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:22:08.151222 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a8e642-1436-40d7-a8b5-5d13265edfbb None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:08.202913 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda44c20-7b4d-40f9-a17e-a4bf4cb31d3a None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:08.231609 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:08.247692 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445d9eb3-ae6e-4099-95d6-673d3d2af317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:08.248398 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445d9eb3-ae6e-4099-95d6-673d3d2af317 None admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:08.248797 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445d9eb3-ae6e-4099-95d6-673d3d2af317 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:08.249075 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445d9eb3-ae6e-4099-95d6-673d3d2af317 None admin] PATH_INFO: `/v3/projects/c02e49a89cdd40d885bfa816f06c58a1/users/31b331b6ddfe46aeae7b52fcbbc27771/roles/1712d85aedaf4189bf4bc89094d888c3` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:08.268860 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445d9eb3-ae6e-4099-95d6-673d3d2af317 None admin] RBAC: Authorizing `identity:create_grant(project_id=c02e49a89cdd40d885bfa816f06c58a1, user_id=31b331b6ddfe46aeae7b52fcbbc27771, role_id=1712d85aedaf4189bf4bc89094d888c3)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:08.270663 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445d9eb3-ae6e-4099-95d6-673d3d2af317 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:08.272286 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a8e642-1436-40d7-a8b5-5d13265edfbb None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:08.315936 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605a5ab7-59f1-43af-bfd8-b13d621228b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:08.316877 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a5ab7-59f1-43af-bfd8-b13d621228b4 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:08.317161 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a5ab7-59f1-43af-bfd8-b13d621228b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:08.317477 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a5ab7-59f1-43af-bfd8-b13d621228b4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:08.328977 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605a5ab7-59f1-43af-bfd8-b13d621228b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:22:08.334164 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-605a5ab7-59f1-43af-bfd8-b13d621228b4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:22:08.334164 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:08.334164 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:08.334164 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:08.334164 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:08.334164 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:08.334164 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:08.334164 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:08.334164 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:08.334164 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:08.334164 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:08.334164 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:08.334164 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:22:08.334164 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:22:08.334164 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:22:08.334164 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:22:08.335101 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:22:08.335101 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:22:08.335101 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:22:08.335101 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:22:08.335101 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:08.335101 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:08.335101 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:08.335101 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:08.335101 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:08.335101 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:08.335101 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:08.335101 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:08.335101 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:08.335101 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:08.335101 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:08.335101 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:08.336687 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:08.336687 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:08.336687 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:08.336687 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:08.336687 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:22:08.336687 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:22:08.336687 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:22:08.336687 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:22:08.336687 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:22:08.336687 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:22:08.336687 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:22:08.336687 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:08.336687 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 300/605] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Aug 24 06:22:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:08.336687 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605a5ab7-59f1-43af-bfd8-b13d621228b4 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:08.348895 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 306/606] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Aug 24 06:22:08 2022] PUT /identity/v3/projects/c02e49a89cdd40d885bfa816f06c58a1/users/31b331b6ddfe46aeae7b52fcbbc27771/roles/1712d85aedaf4189bf4bc89094d888c3 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:22:08.350899 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445d9eb3-ae6e-4099-95d6-673d3d2af317 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:08.485705 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:08.490818 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445d9eb3-ae6e-4099-95d6-673d3d2af317 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:08.568739 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b50e753-b512-4540-910c-52b98f760efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:08.569406 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b50e753-b512-4540-910c-52b98f760efc None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:08.569689 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b50e753-b512-4540-910c-52b98f760efc None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:08.569965 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b50e753-b512-4540-910c-52b98f760efc None admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:08.570777 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b50e753-b512-4540-910c-52b98f760efc None admin] RBAC: Adding query filter params (name=service) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:08.571086 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b50e753-b512-4540-910c-52b98f760efc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:08.572869 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b50e753-b512-4540-910c-52b98f760efc None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:08.583443 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605a5ab7-59f1-43af-bfd8-b13d621228b4 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:08.591696 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 307/607] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Aug 24 06:22:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:08.597366 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b50e753-b512-4540-910c-52b98f760efc None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:08.637493 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:08.643662 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ea23a2-c7a4-4537-b96c-9022afd92a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:08.644357 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea23a2-c7a4-4537-b96c-9022afd92a31 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:08.644637 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea23a2-c7a4-4537-b96c-9022afd92a31 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:08.644911 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ea23a2-c7a4-4537-b96c-9022afd92a31 None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:08.645766 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ea23a2-c7a4-4537-b96c-9022afd92a31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:08.646084 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ea23a2-c7a4-4537-b96c-9022afd92a31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:08.649999 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ea23a2-c7a4-4537-b96c-9022afd92a31 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:08.666940 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 301/608] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 24 06:22:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:08.673164 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ea23a2-c7a4-4537-b96c-9022afd92a31 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:08.791361 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b50e753-b512-4540-910c-52b98f760efc None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:08.834297 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d2b090-f4b8-4c05-924f-23c244993096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:08.835474 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d2b090-f4b8-4c05-924f-23c244993096 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:08.835965 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d2b090-f4b8-4c05-924f-23c244993096 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:08.836463 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d2b090-f4b8-4c05-924f-23c244993096 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:08.837553 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ea23a2-c7a4-4537-b96c-9022afd92a31 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:08.851663 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d2b090-f4b8-4c05-924f-23c244993096 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:08.853310 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d2b090-f4b8-4c05-924f-23c244993096 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:08.870492 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-b1d2b090-f4b8-4c05-924f-23c244993096 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:08.870492 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:08.870492 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:08.870492 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:08.870492 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:08.870492 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:08.870492 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:08.870492 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:08.870492 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:08.870492 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:08.870492 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:08.870492 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:08.870492 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:22:08.870492 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:22:08.870492 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:22:08.870492 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:22:08.871702 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:08.871702 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:08.871702 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:08.871702 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:08.871702 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:08.871702 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:08.871702 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:08.871702 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:08.871702 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:08.871702 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:08.871702 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:08.871702 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:08.871702 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:08.871702 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:08.871702 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:08.871702 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:08.871702 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:22:08.872610 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:22:08.872610 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:08.872610 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:08.872610 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 308/609] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:22:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:22:08.877759 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7334e7-bba1-4c08-9f7d-c56b47214c80 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:08.877759 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7334e7-bba1-4c08-9f7d-c56b47214c80 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:08.877759 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7334e7-bba1-4c08-9f7d-c56b47214c80 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:08.877759 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 309/610] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:22:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:22:08.882055 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7334e7-bba1-4c08-9f7d-c56b47214c80 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:08.895596 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:08.902110 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf74700-8782-477d-917d-a05ef27c3ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:08.903836 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf74700-8782-477d-917d-a05ef27c3ea8 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:08.904805 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf74700-8782-477d-917d-a05ef27c3ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:08.905587 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf74700-8782-477d-917d-a05ef27c3ea8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:08.917465 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf74700-8782-477d-917d-a05ef27c3ea8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:08.924801 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf74700-8782-477d-917d-a05ef27c3ea8 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:08.931971 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-0cf74700-8782-477d-917d-a05ef27c3ea8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:22:08.931971 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:08.931971 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:08.931971 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:08.931971 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:08.931971 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:08.931971 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:08.931971 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:08.931971 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:08.931971 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:08.931971 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:08.931971 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:08.931971 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 06:22:08.931971 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 06:22:08.931971 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 06:22:08.933071 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 06:22:08.933071 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:08.933071 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:08.933071 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:08.933071 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:08.933071 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:08.933071 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:08.933071 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:08.933071 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:08.933071 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:08.933071 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:08.933071 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:08.933071 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:08.933071 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:08.933071 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:08.933071 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:08.933071 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:08.933994 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 06:22:08.933994 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 06:22:08.933994 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:22:08.933994 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:22:08.933994 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:22:08.933994 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:22:08.933994 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:22:08.933994 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:08.937410 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 302/611] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 24 06:22:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:08.938729 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e25777-f8c2-4bee-9bb6-b119ce6bbb44 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:08.939015 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e25777-f8c2-4bee-9bb6-b119ce6bbb44 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:08.939296 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e25777-f8c2-4bee-9bb6-b119ce6bbb44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:08.961519 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-46e25777-f8c2-4bee-9bb6-b119ce6bbb44 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:22:08.962751 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-46e25777-f8c2-4bee-9bb6-b119ce6bbb44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:22:09.022849 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:09.029111 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7334e7-bba1-4c08-9f7d-c56b47214c80 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:09.054779 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 303/612] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:22:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:22:09.056040 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e25777-f8c2-4bee-9bb6-b119ce6bbb44 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:09.089217 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b259f1e3-1c64-4098-9e53-c2c268fc6e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:09.089889 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b259f1e3-1c64-4098-9e53-c2c268fc6e3e None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:09.090228 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b259f1e3-1c64-4098-9e53-c2c268fc6e3e None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:09.090513 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b259f1e3-1c64-4098-9e53-c2c268fc6e3e None admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:09.091699 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b259f1e3-1c64-4098-9e53-c2c268fc6e3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:09.092014 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b259f1e3-1c64-4098-9e53-c2c268fc6e3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:09.093705 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b259f1e3-1c64-4098-9e53-c2c268fc6e3e None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:09.110818 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 310/613] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 24 06:22:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:09.118194 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b259f1e3-1c64-4098-9e53-c2c268fc6e3e None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:09.173255 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e25777-f8c2-4bee-9bb6-b119ce6bbb44 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:09.210655 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075ae777-7875-4941-90f1-9b59c8f2ef5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:09.211617 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ae777-7875-4941-90f1-9b59c8f2ef5f None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:09.212043 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ae777-7875-4941-90f1-9b59c8f2ef5f None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:09.212494 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ae777-7875-4941-90f1-9b59c8f2ef5f None admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:09.213967 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075ae777-7875-4941-90f1-9b59c8f2ef5f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:09.214469 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075ae777-7875-4941-90f1-9b59c8f2ef5f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:09.216055 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075ae777-7875-4941-90f1-9b59c8f2ef5f None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:09.228740 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 304/614] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Aug 24 06:22:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:09.238197 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075ae777-7875-4941-90f1-9b59c8f2ef5f None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:09.283237 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:09.331003 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:09.375439 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075ae777-7875-4941-90f1-9b59c8f2ef5f None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:09.383974 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b259f1e3-1c64-4098-9e53-c2c268fc6e3e None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:09.423736 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca7356c-2093-4f4b-b276-4d8a87818477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:09.424644 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca7356c-2093-4f4b-b276-4d8a87818477 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:09.425167 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca7356c-2093-4f4b-b276-4d8a87818477 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:09.425886 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca7356c-2093-4f4b-b276-4d8a87818477 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:09.435050 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca7356c-2093-4f4b-b276-4d8a87818477 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:09.436538 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca7356c-2093-4f4b-b276-4d8a87818477 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:09.444346 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb723b3a-d239-4d93-8d46-bcf9e180135f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:09.445032 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb723b3a-d239-4d93-8d46-bcf9e180135f None admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:09.445337 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb723b3a-d239-4d93-8d46-bcf9e180135f None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:09.445619 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb723b3a-d239-4d93-8d46-bcf9e180135f None admin] PATH_INFO: `/v3/projects/c02e49a89cdd40d885bfa816f06c58a1/users/f0ba3f03ec15475ab6ebf64994ad742c/roles/1712d85aedaf4189bf4bc89094d888c3` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:09.449976 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-bca7356c-2093-4f4b-b276-4d8a87818477 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 06:22:09.449976 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:09.449976 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:09.449976 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:09.449976 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:09.449976 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:09.449976 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:09.449976 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:09.449976 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:09.449976 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:09.449976 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:09.449976 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:09.449976 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 06:22:09.449976 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 06:22:09.449976 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 06:22:09.449976 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 06:22:09.450901 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:09.450901 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:09.450901 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 06:22:09.450901 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:22:09.450901 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 06:22:09.450901 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:22:09.450901 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:09.450901 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:09.450901 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:09.450901 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:09.450901 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:09.450901 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:09.450901 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:09.450901 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:09.450901 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:09.450901 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:09.450901 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:09.450901 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:09.451852 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:09.451852 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:09.451852 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 06:22:09.451852 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 06:22:09.451852 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 06:22:09.451852 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 06:22:09.451852 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 06:22:09.451852 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 06:22:09.451852 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 06:22:09.451852 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:09.451852 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 311/615] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Aug 24 06:22:09 2022] GET /identity/v3/users/swift => generated 83 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:09.452436 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca7356c-2093-4f4b-b276-4d8a87818477 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:09.465391 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb723b3a-d239-4d93-8d46-bcf9e180135f None admin] RBAC: Authorizing `identity:create_grant(project_id=c02e49a89cdd40d885bfa816f06c58a1, user_id=f0ba3f03ec15475ab6ebf64994ad742c, role_id=1712d85aedaf4189bf4bc89094d888c3)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:09.466948 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb723b3a-d239-4d93-8d46-bcf9e180135f None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:09.545981 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 305/616] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Aug 24 06:22:09 2022] PUT /identity/v3/projects/c02e49a89cdd40d885bfa816f06c58a1/users/f0ba3f03ec15475ab6ebf64994ad742c/roles/1712d85aedaf4189bf4bc89094d888c3 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:22:09.547172 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb723b3a-d239-4d93-8d46-bcf9e180135f None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:09.592358 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca7356c-2093-4f4b-b276-4d8a87818477 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:09.629231 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:09.636644 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28026926-a585-45cf-8557-acbc1a90de95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:09.637433 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28026926-a585-45cf-8557-acbc1a90de95 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:09.640833 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28026926-a585-45cf-8557-acbc1a90de95 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:09.640833 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28026926-a585-45cf-8557-acbc1a90de95 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:09.652287 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28026926-a585-45cf-8557-acbc1a90de95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:22:09.657840 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-28026926-a585-45cf-8557-acbc1a90de95 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:22:09.657840 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:09.657840 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:09.657840 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:09.657840 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:09.657840 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:09.657840 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:09.657840 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:09.657840 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:09.657840 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:09.657840 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:09.657840 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:09.657840 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:22:09.657840 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:22:09.657840 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:22:09.657840 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:22:09.658999 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:22:09.658999 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:22:09.658999 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:22:09.658999 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:22:09.658999 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:09.658999 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:09.658999 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:09.658999 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:09.658999 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:09.658999 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:09.658999 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:09.658999 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:09.658999 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:09.658999 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:09.658999 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:09.658999 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:09.659855 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:09.659855 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:09.659855 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:09.659855 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:09.659855 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:22:09.659855 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:22:09.659855 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:22:09.659855 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:22:09.659855 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:22:09.659855 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:22:09.659855 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:22:09.659855 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:09.660509 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 312/617] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Aug 24 06:22:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:09.661981 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5121afac-cea4-4e98-95b3-cb2955677fb1 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:09.662351 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5121afac-cea4-4e98-95b3-cb2955677fb1 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:09.662753 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5121afac-cea4-4e98-95b3-cb2955677fb1 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:09.664531 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 313/618] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:22:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:22:09.665503 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5121afac-cea4-4e98-95b3-cb2955677fb1 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:09.705750 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:09.721470 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb723b3a-d239-4d93-8d46-bcf9e180135f None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:09.764141 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc458a8-18d9-4fa8-a0f4-9a1032d9f5de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:09.765482 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc458a8-18d9-4fa8-a0f4-9a1032d9f5de None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:09.765868 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc458a8-18d9-4fa8-a0f4-9a1032d9f5de None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:09.766265 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc458a8-18d9-4fa8-a0f4-9a1032d9f5de None admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:09.767610 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc458a8-18d9-4fa8-a0f4-9a1032d9f5de None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:09.768036 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc458a8-18d9-4fa8-a0f4-9a1032d9f5de None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:09.770363 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc458a8-18d9-4fa8-a0f4-9a1032d9f5de None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:09.797504 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 306/619] 10.222.0.47 () {56 vars in 1135 bytes} [Wed Aug 24 06:22:09 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:09.799633 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b6051f-40c3-4dc9-880a-8d0c5cee60f1 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:09.800137 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b6051f-40c3-4dc9-880a-8d0c5cee60f1 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:09.800628 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b6051f-40c3-4dc9-880a-8d0c5cee60f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:09.825789 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5121afac-cea4-4e98-95b3-cb2955677fb1 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:09.827480 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-85b6051f-40c3-4dc9-880a-8d0c5cee60f1 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:22:09.830059 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-85b6051f-40c3-4dc9-880a-8d0c5cee60f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:22:09.869406 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357ce378-9f2a-42cb-9b66-b99c9e99929d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:09.870609 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357ce378-9f2a-42cb-9b66-b99c9e99929d None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:09.870892 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357ce378-9f2a-42cb-9b66-b99c9e99929d None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:09.871166 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357ce378-9f2a-42cb-9b66-b99c9e99929d None admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:09.872204 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357ce378-9f2a-42cb-9b66-b99c9e99929d None admin] RBAC: Adding query filter params (name=service) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:09.872561 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357ce378-9f2a-42cb-9b66-b99c9e99929d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:09.874334 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357ce378-9f2a-42cb-9b66-b99c9e99929d None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:09.884595 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 314/620] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Aug 24 06:22:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:09.886132 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357ce378-9f2a-42cb-9b66-b99c9e99929d None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:09.927347 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 307/621] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:22:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:22:09.929322 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b6051f-40c3-4dc9-880a-8d0c5cee60f1 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:09.982536 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:10.022696 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:10.053382 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b6051f-40c3-4dc9-880a-8d0c5cee60f1 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:10.096155 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36d97b4-b3d4-47ff-bcc7-bbb273af957d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:10.097333 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36d97b4-b3d4-47ff-bcc7-bbb273af957d None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:10.097846 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36d97b4-b3d4-47ff-bcc7-bbb273af957d None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:10.098336 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36d97b4-b3d4-47ff-bcc7-bbb273af957d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:10.098667 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357ce378-9f2a-42cb-9b66-b99c9e99929d None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:10.111422 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36d97b4-b3d4-47ff-bcc7-bbb273af957d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:10.113710 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36d97b4-b3d4-47ff-bcc7-bbb273af957d None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:10.133725 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-e36d97b4-b3d4-47ff-bcc7-bbb273af957d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:10.133725 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:10.133725 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:10.133725 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:10.133725 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:10.133725 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:10.133725 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:10.133725 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:10.133725 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:10.133725 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:10.133725 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:10.133725 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:10.133725 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:22:10.133725 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:22:10.133725 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:22:10.133725 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:22:10.134715 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:10.134715 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:10.134715 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:10.134715 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:10.134715 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:10.134715 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:10.134715 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:10.134715 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:10.134715 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:10.134715 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:10.134715 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:10.134715 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:10.134715 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:10.134715 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:10.134715 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:10.134715 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:10.134715 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:22:10.135703 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:22:10.135703 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:10.135703 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:10.138434 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 308/622] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:22:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:10.144176 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36d97b4-b3d4-47ff-bcc7-bbb273af957d None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:10.185956 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954aedfb-71f8-4c00-b21e-05903eb11114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:10.186617 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954aedfb-71f8-4c00-b21e-05903eb11114 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:10.186900 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954aedfb-71f8-4c00-b21e-05903eb11114 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:10.187172 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954aedfb-71f8-4c00-b21e-05903eb11114 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:10.204229 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954aedfb-71f8-4c00-b21e-05903eb11114 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:10.205972 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954aedfb-71f8-4c00-b21e-05903eb11114 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:10.225294 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-954aedfb-71f8-4c00-b21e-05903eb11114 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:10.225294 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:10.225294 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:10.225294 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:10.225294 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:10.225294 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:10.225294 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:10.225294 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:10.225294 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:10.225294 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:10.225294 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:10.225294 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:10.225294 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:22:10.225294 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:22:10.225294 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:22:10.225294 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:22:10.226493 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:10.226493 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:10.226493 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:10.226493 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:10.226493 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:10.226493 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:10.226493 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:10.226493 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:10.226493 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:10.226493 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:10.226493 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:10.226493 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:10.226493 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:10.226493 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:10.226493 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:10.226493 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:10.226493 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:22:10.227420 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:22:10.227420 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:10.227420 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:10.234689 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 315/623] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:22:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:10.235878 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954aedfb-71f8-4c00-b21e-05903eb11114 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:10.280787 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36d97b4-b3d4-47ff-bcc7-bbb273af957d None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:10.324792 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:10.336425 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77285aa8-07a2-4ed6-8da7-cc847b23a6f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:10.337837 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77285aa8-07a2-4ed6-8da7-cc847b23a6f4 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:10.338126 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77285aa8-07a2-4ed6-8da7-cc847b23a6f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:10.338943 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77285aa8-07a2-4ed6-8da7-cc847b23a6f4 None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:10.339604 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77285aa8-07a2-4ed6-8da7-cc847b23a6f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:10.340528 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77285aa8-07a2-4ed6-8da7-cc847b23a6f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:10.343474 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77285aa8-07a2-4ed6-8da7-cc847b23a6f4 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:10.355941 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954aedfb-71f8-4c00-b21e-05903eb11114 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:10.358168 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:10.365546 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 309/624] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 24 06:22:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:10.365546 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77285aa8-07a2-4ed6-8da7-cc847b23a6f4 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:10.413868 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edee1c24-6842-4556-a41a-4aad2a31ee20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:10.414533 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edee1c24-6842-4556-a41a-4aad2a31ee20 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:10.414886 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edee1c24-6842-4556-a41a-4aad2a31ee20 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:10.415233 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edee1c24-6842-4556-a41a-4aad2a31ee20 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:10.429843 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edee1c24-6842-4556-a41a-4aad2a31ee20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:22:10.436407 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-edee1c24-6842-4556-a41a-4aad2a31ee20 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:22:10.436407 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:10.436407 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:10.436407 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:10.436407 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:10.436407 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:10.436407 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:10.436407 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:10.436407 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:10.436407 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:10.436407 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:10.436407 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:10.436407 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:22:10.436407 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:22:10.436407 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:22:10.436407 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:22:10.437421 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:22:10.437421 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:22:10.437421 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:22:10.437421 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:22:10.437421 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:10.437421 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:10.437421 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:10.437421 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:10.437421 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:10.437421 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:10.437421 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:10.437421 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:10.437421 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:10.437421 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:10.437421 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:10.437421 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:10.438310 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:10.438310 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:10.438310 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:10.438310 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:10.438310 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:22:10.438310 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:22:10.438310 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:22:10.438310 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:22:10.438310 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:22:10.438310 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:22:10.438310 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:22:10.438310 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:10.438971 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 316/625] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Aug 24 06:22:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:10.439378 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edee1c24-6842-4556-a41a-4aad2a31ee20 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:10.497679 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77285aa8-07a2-4ed6-8da7-cc847b23a6f4 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:10.546435 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea59607e-4c5c-4809-96b3-f703950bc0df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:10.547394 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea59607e-4c5c-4809-96b3-f703950bc0df None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:10.547741 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea59607e-4c5c-4809-96b3-f703950bc0df None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:10.548083 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea59607e-4c5c-4809-96b3-f703950bc0df None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:10.549010 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea59607e-4c5c-4809-96b3-f703950bc0df None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:10.549731 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea59607e-4c5c-4809-96b3-f703950bc0df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:10.551358 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea59607e-4c5c-4809-96b3-f703950bc0df None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:10.559837 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edee1c24-6842-4556-a41a-4aad2a31ee20 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:10.567409 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 310/626] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 24 06:22:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:10.568295 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea59607e-4c5c-4809-96b3-f703950bc0df None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:10.607301 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4d9f46-ffd6-4cd6-a4d3-8eb693df3c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:10.608342 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4d9f46-ffd6-4cd6-a4d3-8eb693df3c13 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:10.608826 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4d9f46-ffd6-4cd6-a4d3-8eb693df3c13 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:10.609339 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4d9f46-ffd6-4cd6-a4d3-8eb693df3c13 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:10.624786 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4d9f46-ffd6-4cd6-a4d3-8eb693df3c13 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:10.626601 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4d9f46-ffd6-4cd6-a4d3-8eb693df3c13 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:10.636902 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:10.646761 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-4b4d9f46-ffd6-4cd6-a4d3-8eb693df3c13 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 06:22:10.646761 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:10.646761 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:10.646761 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:10.646761 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:10.646761 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:10.646761 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:10.646761 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:10.646761 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:10.646761 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:10.646761 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:10.646761 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:10.646761 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 06:22:10.646761 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 06:22:10.646761 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 06:22:10.646761 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 06:22:10.656352 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:10.656352 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:10.656352 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 06:22:10.656352 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:22:10.656352 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 06:22:10.656352 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:22:10.656352 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:10.656352 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:10.656352 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:10.656352 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:10.656352 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:10.656352 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:10.656352 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:10.656352 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:10.656352 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:10.656352 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:10.656352 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:10.656352 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:10.657359 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:10.657359 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:10.657359 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 06:22:10.657359 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 06:22:10.657359 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 06:22:10.657359 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 06:22:10.657359 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 06:22:10.657359 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 06:22:10.657359 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 06:22:10.657359 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:10.657359 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 317/627] 10.222.0.47 () {56 vars in 1093 bytes} [Wed Aug 24 06:22:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:10.657359 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4d9f46-ffd6-4cd6-a4d3-8eb693df3c13 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:10.682174 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:10.694729 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea59607e-4c5c-4809-96b3-f703950bc0df None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:10.731907 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f24c49-7675-447c-8939-d7c2fc6956a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:10.732520 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f24c49-7675-447c-8939-d7c2fc6956a6 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:10.732794 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f24c49-7675-447c-8939-d7c2fc6956a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:10.733057 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f24c49-7675-447c-8939-d7c2fc6956a6 None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:10.734174 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f24c49-7675-447c-8939-d7c2fc6956a6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:10.734174 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f24c49-7675-447c-8939-d7c2fc6956a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:10.735568 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f24c49-7675-447c-8939-d7c2fc6956a6 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:10.745589 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 311/628] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Aug 24 06:22:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:10.747253 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f24c49-7675-447c-8939-d7c2fc6956a6 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:10.779131 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4d9f46-ffd6-4cd6-a4d3-8eb693df3c13 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:10.821442 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06a6935-d0b1-4cd9-80c7-e777dc97d543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:10.822104 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06a6935-d0b1-4cd9-80c7-e777dc97d543 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:10.822387 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06a6935-d0b1-4cd9-80c7-e777dc97d543 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:10.822693 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06a6935-d0b1-4cd9-80c7-e777dc97d543 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:10.832081 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06a6935-d0b1-4cd9-80c7-e777dc97d543 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:10.833585 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06a6935-d0b1-4cd9-80c7-e777dc97d543 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:10.846278 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-f06a6935-d0b1-4cd9-80c7-e777dc97d543 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:22:10.846278 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:10.846278 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:10.846278 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:10.846278 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:10.846278 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:10.846278 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:10.846278 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:10.846278 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:10.846278 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:10.846278 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:10.846278 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:10.846278 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 06:22:10.846278 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 06:22:10.846278 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 06:22:10.847300 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 06:22:10.847300 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:10.847300 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:10.847300 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:10.847300 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:10.847300 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:10.847300 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:10.847300 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:10.847300 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:10.847300 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:10.847300 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:10.847300 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:10.847300 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:10.847300 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:10.847300 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:10.847300 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:10.847300 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:10.848226 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 06:22:10.848226 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 06:22:10.848226 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:22:10.848226 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:22:10.848226 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:22:10.848226 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:22:10.848226 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:22:10.848226 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:10.848226 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 318/629] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 24 06:22:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:10.849251 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5bf37e-3b10-4e9a-b78a-1f52974bcce6 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:10.849354 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5bf37e-3b10-4e9a-b78a-1f52974bcce6 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:10.849607 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5bf37e-3b10-4e9a-b78a-1f52974bcce6 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:10.851031 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 319/630] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:22:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:22:10.856445 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5bf37e-3b10-4e9a-b78a-1f52974bcce6 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:10.871494 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f24c49-7675-447c-8939-d7c2fc6956a6 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:10.882786 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:10.910355 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aeb68b-e988-4f75-a592-d5785a0cc2fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:10.910968 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aeb68b-e988-4f75-a592-d5785a0cc2fc None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:10.911243 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aeb68b-e988-4f75-a592-d5785a0cc2fc None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:10.911513 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aeb68b-e988-4f75-a592-d5785a0cc2fc None admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:10.912243 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59aeb68b-e988-4f75-a592-d5785a0cc2fc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:10.912549 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59aeb68b-e988-4f75-a592-d5785a0cc2fc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:10.914013 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59aeb68b-e988-4f75-a592-d5785a0cc2fc None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:10.941433 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 312/631] 10.222.0.47 () {56 vars in 1133 bytes} [Wed Aug 24 06:22:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:10.941433 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aeb68b-e988-4f75-a592-d5785a0cc2fc None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:10.947705 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:10.968295 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5bf37e-3b10-4e9a-b78a-1f52974bcce6 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:11.010850 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb17bfc7-bc28-4292-a4ce-f088bcae21e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:11.011490 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb17bfc7-bc28-4292-a4ce-f088bcae21e3 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:11.011773 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb17bfc7-bc28-4292-a4ce-f088bcae21e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:11.012046 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb17bfc7-bc28-4292-a4ce-f088bcae21e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:11.025539 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb17bfc7-bc28-4292-a4ce-f088bcae21e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:11.027773 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb17bfc7-bc28-4292-a4ce-f088bcae21e3 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:11.043872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-eb17bfc7-bc28-4292-a4ce-f088bcae21e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:11.043872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:11.043872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:11.043872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:11.043872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:11.043872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:11.043872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:11.043872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:11.043872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:11.043872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:11.043872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:11.043872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:11.043872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:22:11.043872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:22:11.043872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:22:11.043872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:22:11.044887 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:11.044887 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:11.044887 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:11.044887 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:11.044887 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:11.044887 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:11.044887 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:11.044887 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:11.044887 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:11.044887 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:11.044887 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:11.044887 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:11.044887 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:11.044887 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:11.044887 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:11.044887 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:11.044887 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:22:11.045803 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:22:11.045803 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:11.045803 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:11.045803 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 320/632] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:22:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:11.046448 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d17a246-191b-4d9d-a89d-f17bd082099e None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:11.046543 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d17a246-191b-4d9d-a89d-f17bd082099e None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:11.046834 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d17a246-191b-4d9d-a89d-f17bd082099e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:11.061802 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aeb68b-e988-4f75-a592-d5785a0cc2fc None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:11.064863 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-2d17a246-191b-4d9d-a89d-f17bd082099e None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:22:11.066128 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-2d17a246-191b-4d9d-a89d-f17bd082099e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:22:11.100738 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d808b2cc-7174-44e7-9378-63f750ce9be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:11.101383 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d808b2cc-7174-44e7-9378-63f750ce9be1 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:11.101659 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d808b2cc-7174-44e7-9378-63f750ce9be1 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:11.101923 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d808b2cc-7174-44e7-9378-63f750ce9be1 None admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:11.102607 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d808b2cc-7174-44e7-9378-63f750ce9be1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:11.102907 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d808b2cc-7174-44e7-9378-63f750ce9be1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:11.104410 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d808b2cc-7174-44e7-9378-63f750ce9be1 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:11.117642 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 313/633] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Aug 24 06:22:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:11.122279 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d808b2cc-7174-44e7-9378-63f750ce9be1 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:11.149586 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 321/634] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:22:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:22:11.150456 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d17a246-191b-4d9d-a89d-f17bd082099e None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:11.155350 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:11.242337 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:11.257670 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d808b2cc-7174-44e7-9378-63f750ce9be1 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:11.299061 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d17a246-191b-4d9d-a89d-f17bd082099e None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:11.302655 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99eeeec-09a2-4cb8-a83a-2838d89018e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:11.302655 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99eeeec-09a2-4cb8-a83a-2838d89018e0 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:11.303190 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99eeeec-09a2-4cb8-a83a-2838d89018e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:11.303190 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99eeeec-09a2-4cb8-a83a-2838d89018e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:11.345946 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9ada9a-0b22-4e14-8934-df48ef12db99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:11.346885 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9ada9a-0b22-4e14-8934-df48ef12db99 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:11.347341 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9ada9a-0b22-4e14-8934-df48ef12db99 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:11.347993 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9ada9a-0b22-4e14-8934-df48ef12db99 None admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:11.350112 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9ada9a-0b22-4e14-8934-df48ef12db99 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:11.350655 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9ada9a-0b22-4e14-8934-df48ef12db99 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:11.352507 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9ada9a-0b22-4e14-8934-df48ef12db99 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:11.373918 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 322/635] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 24 06:22:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:11.375128 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9ada9a-0b22-4e14-8934-df48ef12db99 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:11.401780 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99eeeec-09a2-4cb8-a83a-2838d89018e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:11.403276 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99eeeec-09a2-4cb8-a83a-2838d89018e0 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:11.435725 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:11.493034 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9ada9a-0b22-4e14-8934-df48ef12db99 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:11.516144 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 314/636] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 24 06:22:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 06:22:11.521485 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99eeeec-09a2-4cb8-a83a-2838d89018e0 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:11.522772 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:11.537555 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281c3cc4-a569-4548-8f00-fb3cd12603e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:11.538312 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281c3cc4-a569-4548-8f00-fb3cd12603e1 None admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:11.538717 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281c3cc4-a569-4548-8f00-fb3cd12603e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:11.539001 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281c3cc4-a569-4548-8f00-fb3cd12603e1 None admin] PATH_INFO: `/v3/projects/c02e49a89cdd40d885bfa816f06c58a1/users/02f1cbaf2cd6427585f298c9d58c319a/roles/1712d85aedaf4189bf4bc89094d888c3` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:11.554728 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281c3cc4-a569-4548-8f00-fb3cd12603e1 None admin] RBAC: Authorizing `identity:create_grant(project_id=c02e49a89cdd40d885bfa816f06c58a1, user_id=02f1cbaf2cd6427585f298c9d58c319a, role_id=1712d85aedaf4189bf4bc89094d888c3)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:11.556667 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281c3cc4-a569-4548-8f00-fb3cd12603e1 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:11.628712 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99eeeec-09a2-4cb8-a83a-2838d89018e0 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:11.641212 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 323/637] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Aug 24 06:22:11 2022] PUT /identity/v3/projects/c02e49a89cdd40d885bfa816f06c58a1/users/02f1cbaf2cd6427585f298c9d58c319a/roles/1712d85aedaf4189bf4bc89094d888c3 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:22:11.641212 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281c3cc4-a569-4548-8f00-fb3cd12603e1 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:11.668829 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:11.684526 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0695ff52-08d0-452a-8ce9-14961455180d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:11.685196 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0695ff52-08d0-452a-8ce9-14961455180d None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:11.685527 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0695ff52-08d0-452a-8ce9-14961455180d None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:11.685847 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0695ff52-08d0-452a-8ce9-14961455180d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:11.694878 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0695ff52-08d0-452a-8ce9-14961455180d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:11.696357 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0695ff52-08d0-452a-8ce9-14961455180d None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:11.710197 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-0695ff52-08d0-452a-8ce9-14961455180d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 06:22:11.710197 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:11.710197 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:11.710197 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:11.710197 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:11.710197 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:11.710197 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:11.710197 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:11.710197 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:11.710197 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:11.710197 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:11.710197 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:11.710197 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 06:22:11.710197 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 06:22:11.710197 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 06:22:11.710197 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 06:22:11.711228 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:11.711228 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:11.711228 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 06:22:11.711228 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:22:11.711228 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 06:22:11.711228 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:22:11.711228 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:11.711228 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:11.711228 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:11.711228 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:11.711228 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:11.711228 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:11.711228 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:11.711228 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:11.711228 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:11.711228 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:11.711228 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:11.711228 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:11.712155 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:11.712155 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:11.712155 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 06:22:11.712155 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 06:22:11.712155 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 06:22:11.712155 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 06:22:11.712155 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 06:22:11.712155 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 06:22:11.712155 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 06:22:11.712155 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:11.712155 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 315/638] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 24 06:22:11 2022] GET /identity/v3/users/cinder => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:11.712798 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0695ff52-08d0-452a-8ce9-14961455180d None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:11.783017 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281c3cc4-a569-4548-8f00-fb3cd12603e1 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:11.836200 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0695ff52-08d0-452a-8ce9-14961455180d None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:11.836617 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28418f3a-7fb9-4d90-b47b-e166c95b4699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:11.837313 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28418f3a-7fb9-4d90-b47b-e166c95b4699 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:11.837598 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28418f3a-7fb9-4d90-b47b-e166c95b4699 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:11.837872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28418f3a-7fb9-4d90-b47b-e166c95b4699 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:11.856185 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28418f3a-7fb9-4d90-b47b-e166c95b4699 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:22:11.861280 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-28418f3a-7fb9-4d90-b47b-e166c95b4699 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:22:11.861280 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:11.861280 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:11.861280 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:11.861280 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:11.861280 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:11.861280 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:11.861280 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:11.861280 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:11.861280 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:11.861280 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:11.861280 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:11.861280 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:22:11.861280 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:22:11.861280 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:22:11.861280 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:22:11.862228 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:22:11.862228 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:22:11.862228 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:22:11.862228 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:22:11.862228 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:11.862228 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:11.862228 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:11.862228 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:11.862228 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:11.862228 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:11.862228 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:11.862228 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:11.862228 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:11.862228 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:11.862228 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:11.862228 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:11.863081 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:11.863081 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:11.863081 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:11.863081 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:11.863081 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:22:11.863081 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:22:11.863081 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:22:11.863081 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:22:11.863081 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:22:11.863081 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:22:11.863081 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:22:11.863081 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:11.865576 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 324/639] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Aug 24 06:22:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:11.867106 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787a5c2f-4632-43ea-a1cc-009d4557309c None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:11.867395 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787a5c2f-4632-43ea-a1cc-009d4557309c None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:11.867671 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787a5c2f-4632-43ea-a1cc-009d4557309c None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:11.869107 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 325/640] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:22:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:22:11.870275 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787a5c2f-4632-43ea-a1cc-009d4557309c None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:11.888297 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5d22b7-87b6-429c-a57b-633309f3fe32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:11.888953 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5d22b7-87b6-429c-a57b-633309f3fe32 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:11.889283 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5d22b7-87b6-429c-a57b-633309f3fe32 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:11.889562 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5d22b7-87b6-429c-a57b-633309f3fe32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:11.901595 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5d22b7-87b6-429c-a57b-633309f3fe32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:11.903060 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5d22b7-87b6-429c-a57b-633309f3fe32 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:11.910514 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:11.920307 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-2f5d22b7-87b6-429c-a57b-633309f3fe32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:11.920307 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:11.920307 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:11.920307 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:11.920307 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:11.920307 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:11.920307 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:11.920307 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:11.920307 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:11.920307 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:11.920307 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:11.920307 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:11.920307 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:22:11.920307 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:22:11.920307 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:22:11.920307 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:22:11.933387 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:11.933387 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:11.933387 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:11.933387 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:11.933387 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:11.933387 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:11.933387 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:11.933387 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:11.933387 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:11.933387 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:11.933387 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:11.933387 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:11.933387 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:11.933387 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:11.933387 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:11.933387 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:11.933387 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:22:11.934299 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:22:11.934299 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:11.934299 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:11.934299 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 316/641] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:22:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:11.934299 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1f3b5c-a6d5-412e-86e4-b5590e11ed43 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:11.934299 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1f3b5c-a6d5-412e-86e4-b5590e11ed43 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:11.934299 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1f3b5c-a6d5-412e-86e4-b5590e11ed43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:11.942646 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-ce1f3b5c-a6d5-412e-86e4-b5590e11ed43 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:22:11.944197 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-ce1f3b5c-a6d5-412e-86e4-b5590e11ed43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:22:11.989605 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:12.001599 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787a5c2f-4632-43ea-a1cc-009d4557309c None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:12.040441 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc7a0bb-fd1c-4ed3-9205-1765db8e11ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:12.040965 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc7a0bb-fd1c-4ed3-9205-1765db8e11ee None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:12.042012 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc7a0bb-fd1c-4ed3-9205-1765db8e11ee None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:12.042247 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 317/642] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:22:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:22:12.042412 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc7a0bb-fd1c-4ed3-9205-1765db8e11ee None admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:12.043294 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc7a0bb-fd1c-4ed3-9205-1765db8e11ee None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:12.043602 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc7a0bb-fd1c-4ed3-9205-1765db8e11ee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:12.044063 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1f3b5c-a6d5-412e-86e4-b5590e11ed43 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:12.045151 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc7a0bb-fd1c-4ed3-9205-1765db8e11ee None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:12.066060 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 326/643] 10.222.0.47 () {56 vars in 1137 bytes} [Wed Aug 24 06:22:11 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:12.067145 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc7a0bb-fd1c-4ed3-9205-1765db8e11ee None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:12.186083 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1f3b5c-a6d5-412e-86e4-b5590e11ed43 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:12.193007 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:12.202984 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc7a0bb-fd1c-4ed3-9205-1765db8e11ee None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:12.244090 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cbf411-ca7c-4344-84a7-557c4f6e7e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:12.244741 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664aef64-1d90-4aa2-aebe-415428607d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:12.244866 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cbf411-ca7c-4344-84a7-557c4f6e7e27 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:12.245055 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cbf411-ca7c-4344-84a7-557c4f6e7e27 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:12.245390 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cbf411-ca7c-4344-84a7-557c4f6e7e27 None admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:12.246111 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664aef64-1d90-4aa2-aebe-415428607d19 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:12.246524 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cbf411-ca7c-4344-84a7-557c4f6e7e27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:12.247261 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cbf411-ca7c-4344-84a7-557c4f6e7e27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:12.247669 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664aef64-1d90-4aa2-aebe-415428607d19 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:12.248134 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664aef64-1d90-4aa2-aebe-415428607d19 None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:12.249024 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664aef64-1d90-4aa2-aebe-415428607d19 None admin] RBAC: Adding query filter params (name=service) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:12.249483 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cbf411-ca7c-4344-84a7-557c4f6e7e27 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:12.249759 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664aef64-1d90-4aa2-aebe-415428607d19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:12.251670 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664aef64-1d90-4aa2-aebe-415428607d19 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:12.266954 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 327/644] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 24 06:22:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:12.267652 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 318/645] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Aug 24 06:22:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:12.267944 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cbf411-ca7c-4344-84a7-557c4f6e7e27 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:12.269068 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664aef64-1d90-4aa2-aebe-415428607d19 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:12.314472 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:12.387650 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cbf411-ca7c-4344-84a7-557c4f6e7e27 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:12.395808 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664aef64-1d90-4aa2-aebe-415428607d19 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:12.427053 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cde154-0b14-47d0-bedb-c5230fb75f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:12.427053 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cde154-0b14-47d0-bedb-c5230fb75f00 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:12.427388 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cde154-0b14-47d0-bedb-c5230fb75f00 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:12.427874 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cde154-0b14-47d0-bedb-c5230fb75f00 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:12.437309 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0820dd5-84d1-496d-a4f2-ee5ed97df22f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:12.437957 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0820dd5-84d1-496d-a4f2-ee5ed97df22f None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:12.438273 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0820dd5-84d1-496d-a4f2-ee5ed97df22f None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:12.438569 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0820dd5-84d1-496d-a4f2-ee5ed97df22f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:12.445917 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0820dd5-84d1-496d-a4f2-ee5ed97df22f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:12.447406 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0820dd5-84d1-496d-a4f2-ee5ed97df22f None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:12.457052 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:12.462210 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-f0820dd5-84d1-496d-a4f2-ee5ed97df22f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:22:12.462210 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:12.462210 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:12.462210 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:12.462210 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:12.462210 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:12.462210 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:12.462210 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:12.462210 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:12.462210 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:12.462210 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:12.462210 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:12.462210 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 06:22:12.462210 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 06:22:12.462210 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 06:22:12.463638 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 06:22:12.463638 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:12.463638 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:12.463638 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:12.463638 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:12.463638 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:12.463638 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:12.463638 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:12.463638 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:12.463638 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:12.463638 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:12.463638 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:12.463638 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:12.463638 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:12.463638 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:12.463638 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:12.463638 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:12.464786 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 06:22:12.464786 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 06:22:12.464786 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:22:12.464786 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:22:12.464786 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:22:12.464786 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:22:12.464786 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:22:12.464786 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:12.464786 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 319/646] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 24 06:22:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:12.464786 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0820dd5-84d1-496d-a4f2-ee5ed97df22f None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:12.518171 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cde154-0b14-47d0-bedb-c5230fb75f00 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:12.519648 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cde154-0b14-47d0-bedb-c5230fb75f00 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:12.568837 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:12.580360 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0820dd5-84d1-496d-a4f2-ee5ed97df22f None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:12.623647 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbfdda8-34f8-4a10-8742-1d0f577d0716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:12.624336 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbfdda8-34f8-4a10-8742-1d0f577d0716 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:12.624614 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbfdda8-34f8-4a10-8742-1d0f577d0716 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:12.624903 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbfdda8-34f8-4a10-8742-1d0f577d0716 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:12.636871 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 328/647] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 24 06:22:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 06:22:12.638178 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cde154-0b14-47d0-bedb-c5230fb75f00 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:12.645356 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbfdda8-34f8-4a10-8742-1d0f577d0716 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:12.646869 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbfdda8-34f8-4a10-8742-1d0f577d0716 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:12.667678 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-2dbfdda8-34f8-4a10-8742-1d0f577d0716 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:12.667678 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:12.667678 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:12.667678 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:12.667678 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:12.667678 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:12.667678 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:12.667678 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:12.667678 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:12.667678 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:12.667678 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:12.667678 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:12.667678 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:22:12.667678 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:22:12.667678 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:22:12.667678 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:22:12.669297 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:12.669297 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:12.669297 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:12.669297 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:12.669297 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:12.669297 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:12.669297 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:12.669297 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:12.669297 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:12.669297 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:12.669297 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:12.669297 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:12.669297 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:12.669297 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:12.669297 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:12.669297 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:12.669297 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:22:12.670812 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:22:12.670812 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:12.670812 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:12.673711 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 320/648] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:22:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:12.675412 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbfdda8-34f8-4a10-8742-1d0f577d0716 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:12.708217 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:12.761764 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cde154-0b14-47d0-bedb-c5230fb75f00 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:12.799535 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbfdda8-34f8-4a10-8742-1d0f577d0716 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:12.830646 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad9647c-4ad5-451d-9a50-c0961bb98246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:12.830646 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad9647c-4ad5-451d-9a50-c0961bb98246 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:12.830646 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad9647c-4ad5-451d-9a50-c0961bb98246 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:12.830646 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad9647c-4ad5-451d-9a50-c0961bb98246 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:12.831037 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad9647c-4ad5-451d-9a50-c0961bb98246 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:22:12.831037 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-2ad9647c-4ad5-451d-9a50-c0961bb98246 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:22:12.831037 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:12.831037 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:12.831037 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:12.831037 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:12.831037 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:12.831037 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:12.831037 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:12.831037 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:12.831037 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:12.831037 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:12.831037 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:12.831037 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:22:12.831968 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:22:12.831968 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:22:12.831968 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:22:12.831968 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:22:12.831968 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:22:12.831968 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:22:12.831968 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:22:12.831968 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:12.831968 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:12.831968 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:12.831968 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:12.831968 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:12.831968 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:12.831968 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:12.831968 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:12.831968 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:12.831968 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:12.833075 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:12.833075 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:12.833075 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:12.833075 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:12.833075 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:12.833075 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:12.833075 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:22:12.833075 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:22:12.833075 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:22:12.833075 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:22:12.833075 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:22:12.833075 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:22:12.833075 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:22:12.833075 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:12.833075 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 329/649] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Aug 24 06:22:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:12.833987 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad9647c-4ad5-451d-9a50-c0961bb98246 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:12.840812 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03cf8c7-f602-48e6-b47f-c459f10bba08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:12.841763 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03cf8c7-f602-48e6-b47f-c459f10bba08 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:12.841872 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03cf8c7-f602-48e6-b47f-c459f10bba08 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:12.842145 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03cf8c7-f602-48e6-b47f-c459f10bba08 None admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:12.842830 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03cf8c7-f602-48e6-b47f-c459f10bba08 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:12.843130 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03cf8c7-f602-48e6-b47f-c459f10bba08 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:12.844757 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03cf8c7-f602-48e6-b47f-c459f10bba08 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:12.861579 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 321/650] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Aug 24 06:22:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:12.861579 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03cf8c7-f602-48e6-b47f-c459f10bba08 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:12.866625 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:12.936677 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad9647c-4ad5-451d-9a50-c0961bb98246 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:12.950601 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:12.967453 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03cf8c7-f602-48e6-b47f-c459f10bba08 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:12.973642 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24658ed-f822-4873-9c7a-dc89091c662a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:12.973642 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24658ed-f822-4873-9c7a-dc89091c662a None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:12.973642 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24658ed-f822-4873-9c7a-dc89091c662a None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:12.973642 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24658ed-f822-4873-9c7a-dc89091c662a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:12.984306 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24658ed-f822-4873-9c7a-dc89091c662a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:12.986050 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24658ed-f822-4873-9c7a-dc89091c662a None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:13.001255 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-b24658ed-f822-4873-9c7a-dc89091c662a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:13.001255 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:13.001255 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:13.001255 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:13.001255 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:13.001255 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:13.001255 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:13.001255 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:13.001255 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:13.001255 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:13.001255 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:13.001255 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:13.001255 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:22:13.001255 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:22:13.001255 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:22:13.001255 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:22:13.002539 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:13.002539 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:13.002539 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:13.002539 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:13.002539 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:13.002539 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:13.002539 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:13.002539 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:13.002539 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:13.002539 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:13.002539 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:13.002539 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:13.002539 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:13.002539 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:13.002539 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:13.002539 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:13.002539 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:22:13.003620 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:22:13.003620 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:13.003620 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:13.003833 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01e351a-abab-491a-a334-eb40892a4726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:13.003833 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01e351a-abab-491a-a334-eb40892a4726 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:13.003833 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01e351a-abab-491a-a334-eb40892a4726 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:13.004065 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01e351a-abab-491a-a334-eb40892a4726 None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:13.004515 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01e351a-abab-491a-a334-eb40892a4726 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:13.004817 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01e351a-abab-491a-a334-eb40892a4726 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:13.009235 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01e351a-abab-491a-a334-eb40892a4726 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:13.010880 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 330/651] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:22:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:13.011748 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24658ed-f822-4873-9c7a-dc89091c662a None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:13.019375 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 322/652] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 24 06:22:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:13.022474 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01e351a-abab-491a-a334-eb40892a4726 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:13.098305 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:13.118763 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24658ed-f822-4873-9c7a-dc89091c662a None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:13.140985 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01e351a-abab-491a-a334-eb40892a4726 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:13.158928 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec26f9c3-1bd9-4e35-9a37-0d576602db68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:13.159534 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec26f9c3-1bd9-4e35-9a37-0d576602db68 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:13.159811 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec26f9c3-1bd9-4e35-9a37-0d576602db68 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:13.160079 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec26f9c3-1bd9-4e35-9a37-0d576602db68 None admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:13.160736 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec26f9c3-1bd9-4e35-9a37-0d576602db68 None admin] RBAC: Adding query filter params (name=service) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:13.161213 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec26f9c3-1bd9-4e35-9a37-0d576602db68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:13.162492 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec26f9c3-1bd9-4e35-9a37-0d576602db68 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:13.175061 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 331/653] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Aug 24 06:22:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:13.175061 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec26f9c3-1bd9-4e35-9a37-0d576602db68 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:13.182898 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910cba05-5fa1-4eb3-9e99-939ca7124e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:13.183542 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910cba05-5fa1-4eb3-9e99-939ca7124e78 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:13.183818 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910cba05-5fa1-4eb3-9e99-939ca7124e78 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:13.184166 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910cba05-5fa1-4eb3-9e99-939ca7124e78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:13.184895 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910cba05-5fa1-4eb3-9e99-939ca7124e78 None admin] RBAC: Adding query filter params (role.id=1712d85aedaf4189bf4bc89094d888c3, scope.project.id=c02e49a89cdd40d885bfa816f06c58a1, user.id=1f4f49c1b08c460f8ba6c451c9c92c2a) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:13.185220 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910cba05-5fa1-4eb3-9e99-939ca7124e78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:13.186653 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910cba05-5fa1-4eb3-9e99-939ca7124e78 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:13.192985 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 323/654] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 24 06:22:13 2022] GET /identity/v3/role_assignments?user.id=1f4f49c1b08c460f8ba6c451c9c92c2a&scope.project.id=c02e49a89cdd40d885bfa816f06c58a1&role.id=1712d85aedaf4189bf4bc89094d888c3 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:13.198827 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910cba05-5fa1-4eb3-9e99-939ca7124e78 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:13.205075 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:13.301623 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec26f9c3-1bd9-4e35-9a37-0d576602db68 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:13.344461 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b075d8-b8ce-40fb-8585-f2cbeee970bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:13.345073 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b075d8-b8ce-40fb-8585-f2cbeee970bf None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:13.345391 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b075d8-b8ce-40fb-8585-f2cbeee970bf None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:13.345677 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b075d8-b8ce-40fb-8585-f2cbeee970bf None admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:13.346282 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b075d8-b8ce-40fb-8585-f2cbeee970bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:13.346577 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b075d8-b8ce-40fb-8585-f2cbeee970bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:13.348062 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b075d8-b8ce-40fb-8585-f2cbeee970bf None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:13.361914 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910cba05-5fa1-4eb3-9e99-939ca7124e78 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:13.362865 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 332/655] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 24 06:22:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:13.363929 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b075d8-b8ce-40fb-8585-f2cbeee970bf None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:13.395466 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:13.404215 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab6642d-4e0a-4d30-a05a-db79899e18f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:13.404858 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab6642d-4e0a-4d30-a05a-db79899e18f1 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:13.405130 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab6642d-4e0a-4d30-a05a-db79899e18f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:13.405431 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab6642d-4e0a-4d30-a05a-db79899e18f1 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:13.414962 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab6642d-4e0a-4d30-a05a-db79899e18f1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:13.417434 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab6642d-4e0a-4d30-a05a-db79899e18f1 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:13.430710 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-eab6642d-4e0a-4d30-a05a-db79899e18f1 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 06:22:13.430710 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:13.430710 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:13.430710 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:13.430710 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:13.430710 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:13.430710 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:13.430710 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:13.430710 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:13.430710 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:13.430710 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:13.430710 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:13.430710 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 06:22:13.430710 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 06:22:13.430710 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 06:22:13.430710 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 06:22:13.431800 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:13.431800 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:13.431800 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 06:22:13.431800 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:22:13.431800 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 06:22:13.431800 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:22:13.431800 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:13.431800 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:13.431800 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:13.431800 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:13.431800 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:13.431800 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:13.431800 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:13.431800 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:13.431800 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:13.431800 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:13.431800 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:13.431800 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:13.433832 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:13.433832 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:13.433832 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 06:22:13.433832 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 06:22:13.433832 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 06:22:13.433832 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 06:22:13.433832 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 06:22:13.433832 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 06:22:13.433832 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 06:22:13.433832 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:13.435085 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 324/656] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 24 06:22:13 2022] GET /identity/v3/users/glance => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:13.435748 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab6642d-4e0a-4d30-a05a-db79899e18f1 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:13.480471 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:13.499560 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b075d8-b8ce-40fb-8585-f2cbeee970bf None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:13.554112 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c65d688-761c-47be-8d3e-188cbb6456b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:13.554800 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65d688-761c-47be-8d3e-188cbb6456b7 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:13.555087 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65d688-761c-47be-8d3e-188cbb6456b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:13.555363 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65d688-761c-47be-8d3e-188cbb6456b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:13.564416 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab6642d-4e0a-4d30-a05a-db79899e18f1 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:13.568943 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c65d688-761c-47be-8d3e-188cbb6456b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:13.570304 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c65d688-761c-47be-8d3e-188cbb6456b7 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:13.585532 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-3c65d688-761c-47be-8d3e-188cbb6456b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:13.585532 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:13.585532 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:13.585532 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:13.585532 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:13.585532 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:13.585532 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:13.585532 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:13.585532 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:13.585532 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:13.585532 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:13.585532 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:13.585532 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:22:13.585532 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:22:13.585532 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:22:13.585532 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:22:13.586625 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:13.586625 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:13.586625 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:13.586625 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:13.586625 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:13.586625 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:13.586625 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:13.586625 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:13.586625 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:13.586625 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:13.586625 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:13.586625 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:13.586625 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:13.586625 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:13.586625 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:13.586625 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:13.586625 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:22:13.588197 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:22:13.588197 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:13.588197 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:13.588197 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 333/657] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:22:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:13.588197 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c65d688-761c-47be-8d3e-188cbb6456b7 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:13.613983 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc730c8f-f55f-4db3-8f45-dfc99e9b0c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:13.615677 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc730c8f-f55f-4db3-8f45-dfc99e9b0c7e None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:13.616307 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc730c8f-f55f-4db3-8f45-dfc99e9b0c7e None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:13.616854 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc730c8f-f55f-4db3-8f45-dfc99e9b0c7e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:13.629161 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc730c8f-f55f-4db3-8f45-dfc99e9b0c7e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:13.630913 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc730c8f-f55f-4db3-8f45-dfc99e9b0c7e None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:13.647266 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-bc730c8f-f55f-4db3-8f45-dfc99e9b0c7e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:22:13.647266 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:13.647266 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:13.647266 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:13.647266 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:13.647266 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:13.647266 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:13.647266 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:13.647266 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:13.647266 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:13.647266 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:13.647266 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:13.647266 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 06:22:13.647266 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 06:22:13.647266 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 06:22:13.648696 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 06:22:13.648696 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:13.648696 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:13.648696 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:13.648696 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:13.648696 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:13.648696 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:13.648696 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:13.648696 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:13.648696 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:13.648696 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:13.648696 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:13.648696 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:13.648696 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:13.648696 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:13.648696 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:13.648696 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:13.649836 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 06:22:13.649836 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 06:22:13.649836 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:22:13.649836 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:22:13.649836 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:22:13.649836 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:22:13.649836 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:22:13.649836 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:13.656177 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 325/658] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 24 06:22:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:13.657579 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc730c8f-f55f-4db3-8f45-dfc99e9b0c7e None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:13.714030 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:13.719429 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c65d688-761c-47be-8d3e-188cbb6456b7 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:13.768347 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc730c8f-f55f-4db3-8f45-dfc99e9b0c7e None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:13.770103 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b321bf54-a66c-4b01-b66b-cc4ff96ae9b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:13.771084 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b321bf54-a66c-4b01-b66b-cc4ff96ae9b2 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:13.771566 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b321bf54-a66c-4b01-b66b-cc4ff96ae9b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:13.772048 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b321bf54-a66c-4b01-b66b-cc4ff96ae9b2 None admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:13.773669 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b321bf54-a66c-4b01-b66b-cc4ff96ae9b2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:13.774206 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b321bf54-a66c-4b01-b66b-cc4ff96ae9b2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:13.777421 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b321bf54-a66c-4b01-b66b-cc4ff96ae9b2 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:13.793428 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:13.805844 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df34e9f-240e-4c83-9f9a-64b40df5ac59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:13.806462 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df34e9f-240e-4c83-9f9a-64b40df5ac59 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:13.806869 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df34e9f-240e-4c83-9f9a-64b40df5ac59 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:13.807303 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df34e9f-240e-4c83-9f9a-64b40df5ac59 None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:13.808100 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df34e9f-240e-4c83-9f9a-64b40df5ac59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:13.808410 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df34e9f-240e-4c83-9f9a-64b40df5ac59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:13.809867 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df34e9f-240e-4c83-9f9a-64b40df5ac59 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:13.813583 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 334/659] 10.222.0.47 () {56 vars in 1137 bytes} [Wed Aug 24 06:22:13 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:13.815326 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b321bf54-a66c-4b01-b66b-cc4ff96ae9b2 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:13.831723 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 326/660] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 24 06:22:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:13.831723 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f46eed-e424-4e44-bce3-f7b048853078 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:13.831723 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f46eed-e424-4e44-bce3-f7b048853078 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:13.831723 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f46eed-e424-4e44-bce3-f7b048853078 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:13.831723 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 327/661] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:22:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:22:13.831723 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f46eed-e424-4e44-bce3-f7b048853078 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:13.942312 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b321bf54-a66c-4b01-b66b-cc4ff96ae9b2 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:13.949878 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f46eed-e424-4e44-bce3-f7b048853078 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:13.960613 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:13.984979 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd095d6-b81a-45af-96dc-0b106498efed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:13.985621 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd095d6-b81a-45af-96dc-0b106498efed None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:13.985900 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd095d6-b81a-45af-96dc-0b106498efed None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:13.986171 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd095d6-b81a-45af-96dc-0b106498efed None admin] PATH_INFO: `/v3/projects` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:13.986777 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c02fc0-2088-4778-b1f1-84a0ea0dfd0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:13.986907 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd095d6-b81a-45af-96dc-0b106498efed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:13.987150 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd095d6-b81a-45af-96dc-0b106498efed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:13.987397 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c02fc0-2088-4778-b1f1-84a0ea0dfd0f None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:13.987741 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c02fc0-2088-4778-b1f1-84a0ea0dfd0f None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:13.988019 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c02fc0-2088-4778-b1f1-84a0ea0dfd0f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:13.988579 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd095d6-b81a-45af-96dc-0b106498efed None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:14.008418 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 335/662] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Aug 24 06:22:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:14.008418 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd095d6-b81a-45af-96dc-0b106498efed None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:14.062327 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c02fc0-2088-4778-b1f1-84a0ea0dfd0f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:14.064017 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c02fc0-2088-4778-b1f1-84a0ea0dfd0f None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:14.078064 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:14.128618 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd095d6-b81a-45af-96dc-0b106498efed None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:14.167416 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2209dd-bc6b-4478-aa97-45d849d0e417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:14.167971 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2209dd-bc6b-4478-aa97-45d849d0e417 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:14.168247 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2209dd-bc6b-4478-aa97-45d849d0e417 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:14.168565 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2209dd-bc6b-4478-aa97-45d849d0e417 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:14.177432 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 328/663] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 24 06:22:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 06:22:14.177432 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b3441f-3032-4d91-9bfe-ac2c267cb88d None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:14.177432 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b3441f-3032-4d91-9bfe-ac2c267cb88d None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:14.177432 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b3441f-3032-4d91-9bfe-ac2c267cb88d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:14.179524 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2209dd-bc6b-4478-aa97-45d849d0e417 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:14.180879 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2209dd-bc6b-4478-aa97-45d849d0e417 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:14.194143 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-3e2209dd-bc6b-4478-aa97-45d849d0e417 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 06:22:14.194143 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:14.194143 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:14.194143 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:14.194143 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:14.194143 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:14.194143 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:14.194143 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:14.194143 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:14.194143 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:14.194143 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:14.194143 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:14.194143 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 06:22:14.194143 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 06:22:14.194143 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 06:22:14.194143 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 06:22:14.195131 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:14.195131 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:14.195131 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 06:22:14.195131 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:22:14.195131 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 06:22:14.195131 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:22:14.195131 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:14.195131 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:14.195131 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:14.195131 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:14.195131 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:14.195131 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:14.195131 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:14.195131 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:14.195131 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:14.195131 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:14.195131 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:14.195131 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:14.196080 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:14.196080 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:14.196080 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 06:22:14.196080 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 06:22:14.196080 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 06:22:14.196080 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 06:22:14.196080 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 06:22:14.196080 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 06:22:14.196080 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 06:22:14.196080 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:14.197142 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-08b3441f-3032-4d91-9bfe-ac2c267cb88d None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:22:14.197142 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-08b3441f-3032-4d91-9bfe-ac2c267cb88d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:22:14.198609 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:14.200600 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 336/664] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Aug 24 06:22:14 2022] GET /identity/v3/users/neutron => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:14.200742 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2209dd-bc6b-4478-aa97-45d849d0e417 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:14.270270 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 329/665] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:22:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:22:14.271272 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b3441f-3032-4d91-9bfe-ac2c267cb88d None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:14.320551 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2209dd-bc6b-4478-aa97-45d849d0e417 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:14.344627 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:14.367232 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f2e029-d33c-413d-b678-b81c7688382d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:14.368431 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f2e029-d33c-413d-b678-b81c7688382d None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:14.368916 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f2e029-d33c-413d-b678-b81c7688382d None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:14.370714 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f2e029-d33c-413d-b678-b81c7688382d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:14.371550 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f2e029-d33c-413d-b678-b81c7688382d None admin] RBAC: Adding query filter params (role.id=1712d85aedaf4189bf4bc89094d888c3, scope.project.id=c02e49a89cdd40d885bfa816f06c58a1, user.id=36639dce4f97412f89d71367f650b901) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:14.371865 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f2e029-d33c-413d-b678-b81c7688382d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:14.373371 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f2e029-d33c-413d-b678-b81c7688382d None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:14.384193 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 337/666] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 24 06:22:14 2022] GET /identity/v3/role_assignments?user.id=36639dce4f97412f89d71367f650b901&scope.project.id=c02e49a89cdd40d885bfa816f06c58a1&role.id=1712d85aedaf4189bf4bc89094d888c3 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:14.386273 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f2e029-d33c-413d-b678-b81c7688382d None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:14.428001 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b3441f-3032-4d91-9bfe-ac2c267cb88d None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:14.471407 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a3fab3-80e0-4d62-992c-f69e6f43b78d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:14.472881 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a3fab3-80e0-4d62-992c-f69e6f43b78d None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:14.473357 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a3fab3-80e0-4d62-992c-f69e6f43b78d None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:14.473789 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a3fab3-80e0-4d62-992c-f69e6f43b78d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:14.483360 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:14.489308 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a3fab3-80e0-4d62-992c-f69e6f43b78d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:14.491018 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a3fab3-80e0-4d62-992c-f69e6f43b78d None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:14.509887 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-a0a3fab3-80e0-4d62-992c-f69e6f43b78d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:14.509887 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:14.509887 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:14.509887 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:14.509887 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:14.509887 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:14.509887 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:14.509887 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:14.509887 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:14.509887 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:14.509887 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:14.509887 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:14.509887 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:22:14.509887 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:22:14.509887 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:22:14.509887 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:22:14.526620 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:14.526620 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:14.526620 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:14.526620 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:14.526620 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:14.526620 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:14.526620 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:14.526620 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:14.526620 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:14.526620 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:14.526620 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:14.526620 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:14.526620 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:14.526620 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:14.526620 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:14.526620 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:14.526620 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:22:14.527628 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:22:14.527628 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:14.527628 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:14.527628 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 330/667] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:22:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:14.527628 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a3fab3-80e0-4d62-992c-f69e6f43b78d None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:14.575625 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f2e029-d33c-413d-b678-b81c7688382d None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:14.616821 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7410aff4-d4c9-40ae-9b5a-82749f9521e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:14.616821 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7410aff4-d4c9-40ae-9b5a-82749f9521e3 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:14.617263 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7410aff4-d4c9-40ae-9b5a-82749f9521e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:14.617512 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7410aff4-d4c9-40ae-9b5a-82749f9521e3 None admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:14.620983 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7410aff4-d4c9-40ae-9b5a-82749f9521e3 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:14.620983 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7410aff4-d4c9-40ae-9b5a-82749f9521e3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:14.620983 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7410aff4-d4c9-40ae-9b5a-82749f9521e3 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:14.645280 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 338/668] 10.222.0.47 () {56 vars in 1139 bytes} [Wed Aug 24 06:22:14 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:14.646449 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7410aff4-d4c9-40ae-9b5a-82749f9521e3 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:14.680421 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a3fab3-80e0-4d62-992c-f69e6f43b78d None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:14.711523 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:14.730909 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d7dedd-e77a-4e9f-8c78-3b4a38a21eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:14.731598 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d7dedd-e77a-4e9f-8c78-3b4a38a21eeb None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:14.731976 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d7dedd-e77a-4e9f-8c78-3b4a38a21eeb None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:14.732345 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d7dedd-e77a-4e9f-8c78-3b4a38a21eeb None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:14.733101 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d7dedd-e77a-4e9f-8c78-3b4a38a21eeb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:14.733538 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d7dedd-e77a-4e9f-8c78-3b4a38a21eeb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:14.734995 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d7dedd-e77a-4e9f-8c78-3b4a38a21eeb None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:14.748703 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 331/669] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 24 06:22:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:14.749839 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d7dedd-e77a-4e9f-8c78-3b4a38a21eeb None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:14.799395 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7410aff4-d4c9-40ae-9b5a-82749f9521e3 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:14.816461 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:14.846568 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792492e5-7036-4b1c-b335-a5bf4786eb63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:14.847213 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792492e5-7036-4b1c-b335-a5bf4786eb63 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:14.847501 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792492e5-7036-4b1c-b335-a5bf4786eb63 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:14.848244 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792492e5-7036-4b1c-b335-a5bf4786eb63 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:14.863445 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792492e5-7036-4b1c-b335-a5bf4786eb63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:22:14.869981 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-792492e5-7036-4b1c-b335-a5bf4786eb63 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:22:14.869981 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:14.869981 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:14.869981 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:14.869981 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:14.869981 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:14.869981 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:14.869981 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:14.869981 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:14.869981 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:14.869981 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:14.869981 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:14.869981 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:22:14.869981 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:22:14.869981 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:22:14.869981 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:22:14.871170 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:22:14.871170 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:22:14.871170 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:22:14.871170 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:22:14.871170 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:14.871170 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:14.871170 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:14.871170 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:14.871170 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:14.871170 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:14.871170 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:14.871170 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:14.871170 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:14.871170 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:14.871170 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:14.871170 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:14.872705 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:14.872705 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:14.872705 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:14.872705 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:14.872705 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:22:14.872705 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:22:14.872705 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:22:14.872705 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:22:14.872705 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:22:14.872705 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:22:14.872705 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:22:14.872705 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:14.876182 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 339/670] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Aug 24 06:22:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:14.877069 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792492e5-7036-4b1c-b335-a5bf4786eb63 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:14.883606 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d7dedd-e77a-4e9f-8c78-3b4a38a21eeb None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:14.935116 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626f9f39-7578-4211-997a-c555972718e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:14.935779 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f9f39-7578-4211-997a-c555972718e0 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:14.936061 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f9f39-7578-4211-997a-c555972718e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:14.936359 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f9f39-7578-4211-997a-c555972718e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:14.987661 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:14.991924 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792492e5-7036-4b1c-b335-a5bf4786eb63 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:15.021691 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626f9f39-7578-4211-997a-c555972718e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:15.023527 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626f9f39-7578-4211-997a-c555972718e0 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:15.032152 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bb84e3-1473-4b47-b6f9-c766c790a898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:15.032792 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bb84e3-1473-4b47-b6f9-c766c790a898 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:15.033079 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bb84e3-1473-4b47-b6f9-c766c790a898 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:15.033383 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bb84e3-1473-4b47-b6f9-c766c790a898 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:15.040186 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bb84e3-1473-4b47-b6f9-c766c790a898 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:15.042568 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bb84e3-1473-4b47-b6f9-c766c790a898 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:15.059788 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-c6bb84e3-1473-4b47-b6f9-c766c790a898 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:22:15.059788 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:15.059788 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:15.059788 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:15.059788 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:15.059788 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:15.059788 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:15.059788 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:15.059788 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:15.059788 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:15.059788 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:15.059788 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:15.059788 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 06:22:15.059788 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 06:22:15.059788 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 06:22:15.060810 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 06:22:15.060810 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:15.060810 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:15.060810 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:15.060810 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:15.060810 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:15.060810 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:15.060810 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:15.060810 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:15.060810 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:15.060810 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:15.060810 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:15.060810 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:15.060810 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:15.060810 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:15.060810 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:15.060810 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:15.061763 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 06:22:15.061763 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 06:22:15.061763 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:22:15.061763 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:22:15.061763 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:22:15.061763 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:22:15.061763 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:22:15.061763 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:15.062637 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 340/671] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 24 06:22:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:15.063804 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bb84e3-1473-4b47-b6f9-c766c790a898 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:15.084148 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:15.152245 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 332/672] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 24 06:22:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 06:22:15.153839 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626f9f39-7578-4211-997a-c555972718e0 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:15.206073 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bb84e3-1473-4b47-b6f9-c766c790a898 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:15.254694 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad30ed42-7875-4287-87b4-f582f058b61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:15.255246 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad30ed42-7875-4287-87b4-f582f058b61e None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:15.255526 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad30ed42-7875-4287-87b4-f582f058b61e None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:15.255792 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad30ed42-7875-4287-87b4-f582f058b61e None admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:15.256591 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad30ed42-7875-4287-87b4-f582f058b61e None admin] RBAC: Adding query filter params (name=service) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:15.257045 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad30ed42-7875-4287-87b4-f582f058b61e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:15.258706 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad30ed42-7875-4287-87b4-f582f058b61e None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:15.275878 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 341/673] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Aug 24 06:22:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:15.277000 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad30ed42-7875-4287-87b4-f582f058b61e None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:15.287654 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626f9f39-7578-4211-997a-c555972718e0 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:15.305400 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:15.329628 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663a340d-15a1-4d59-bc85-b07e5d44dcb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:15.330670 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663a340d-15a1-4d59-bc85-b07e5d44dcb5 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:15.331270 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663a340d-15a1-4d59-bc85-b07e5d44dcb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:15.331791 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663a340d-15a1-4d59-bc85-b07e5d44dcb5 None admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:15.332799 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663a340d-15a1-4d59-bc85-b07e5d44dcb5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:15.333436 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663a340d-15a1-4d59-bc85-b07e5d44dcb5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:15.335150 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663a340d-15a1-4d59-bc85-b07e5d44dcb5 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:15.351658 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 333/674] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Aug 24 06:22:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:15.353045 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663a340d-15a1-4d59-bc85-b07e5d44dcb5 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:15.372568 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:15.415452 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad30ed42-7875-4287-87b4-f582f058b61e None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:15.459520 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf7822b-d90d-4fa3-93f7-032e62512548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:15.460655 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf7822b-d90d-4fa3-93f7-032e62512548 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:15.460941 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf7822b-d90d-4fa3-93f7-032e62512548 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:15.461250 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf7822b-d90d-4fa3-93f7-032e62512548 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:15.474967 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf7822b-d90d-4fa3-93f7-032e62512548 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:15.476413 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf7822b-d90d-4fa3-93f7-032e62512548 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:15.495686 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-ccf7822b-d90d-4fa3-93f7-032e62512548 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:15.495686 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:15.495686 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:15.495686 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:15.495686 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:15.495686 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:15.495686 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:15.495686 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:15.495686 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:15.495686 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:15.495686 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:15.495686 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:15.495686 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:22:15.495686 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:22:15.495686 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:22:15.495686 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:22:15.496684 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:15.496684 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:15.496684 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:15.496684 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:15.496684 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:15.496684 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:15.496684 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:15.496684 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:15.496684 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:15.496684 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:15.496684 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:15.496684 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:15.496684 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:15.496684 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:15.496684 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:15.496684 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:15.496684 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:22:15.497609 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:22:15.497609 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:15.497609 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:15.497609 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 342/675] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:22:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:15.498267 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf7822b-d90d-4fa3-93f7-032e62512548 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:15.530001 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663a340d-15a1-4d59-bc85-b07e5d44dcb5 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:15.567620 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dcdc25-509f-4cb2-9fac-c845d54727df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:15.568275 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dcdc25-509f-4cb2-9fac-c845d54727df None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:15.568553 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dcdc25-509f-4cb2-9fac-c845d54727df None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:15.568827 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dcdc25-509f-4cb2-9fac-c845d54727df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:15.579614 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:15.581151 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dcdc25-509f-4cb2-9fac-c845d54727df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:15.582725 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dcdc25-509f-4cb2-9fac-c845d54727df None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:15.598259 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-52dcdc25-509f-4cb2-9fac-c845d54727df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:15.598259 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:15.598259 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:15.598259 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:15.598259 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:15.598259 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:15.598259 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:15.598259 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:15.598259 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:15.598259 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:15.598259 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:15.598259 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:15.598259 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:22:15.598259 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:22:15.598259 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:22:15.598259 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:22:15.606528 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:15.606528 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:15.606528 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:15.606528 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:15.606528 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:15.606528 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:15.606528 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:15.606528 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:15.606528 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:15.606528 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:15.606528 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:15.606528 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:15.606528 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:15.606528 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:15.606528 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:15.606528 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:15.606528 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:22:15.607411 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:22:15.607411 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:15.607411 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:15.607411 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 334/676] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:22:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:15.607411 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a1e9cc-eedc-4ab0-bd62-aada30cffac5 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:15.607411 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a1e9cc-eedc-4ab0-bd62-aada30cffac5 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:15.607411 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a1e9cc-eedc-4ab0-bd62-aada30cffac5 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:15.612450 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 335/677] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:22:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:22:15.612450 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a1e9cc-eedc-4ab0-bd62-aada30cffac5 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:15.620452 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf7822b-d90d-4fa3-93f7-032e62512548 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:15.659505 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0c13b3-0e7f-4682-a2fa-1b94b69aa991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:15.660573 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0c13b3-0e7f-4682-a2fa-1b94b69aa991 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:15.661028 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0c13b3-0e7f-4682-a2fa-1b94b69aa991 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:15.661504 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0c13b3-0e7f-4682-a2fa-1b94b69aa991 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:15.662408 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0c13b3-0e7f-4682-a2fa-1b94b69aa991 None admin] RBAC: Adding query filter params (role.id=1712d85aedaf4189bf4bc89094d888c3, scope.project.id=c02e49a89cdd40d885bfa816f06c58a1, user.id=31b331b6ddfe46aeae7b52fcbbc27771) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:15.662950 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0c13b3-0e7f-4682-a2fa-1b94b69aa991 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:15.664579 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0c13b3-0e7f-4682-a2fa-1b94b69aa991 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:15.684353 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 343/678] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 24 06:22:15 2022] GET /identity/v3/role_assignments?user.id=31b331b6ddfe46aeae7b52fcbbc27771&scope.project.id=c02e49a89cdd40d885bfa816f06c58a1&role.id=1712d85aedaf4189bf4bc89094d888c3 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:15.685888 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0c13b3-0e7f-4682-a2fa-1b94b69aa991 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:15.698932 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:15.755719 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a1e9cc-eedc-4ab0-bd62-aada30cffac5 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:15.797122 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5e39e4-f6d0-4f15-93ff-54132cf5a55e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:15.797720 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e39e4-f6d0-4f15-93ff-54132cf5a55e None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:15.798007 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e39e4-f6d0-4f15-93ff-54132cf5a55e None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:15.798295 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e39e4-f6d0-4f15-93ff-54132cf5a55e None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:15.798963 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5e39e4-f6d0-4f15-93ff-54132cf5a55e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:15.799271 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5e39e4-f6d0-4f15-93ff-54132cf5a55e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:15.800667 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5e39e4-f6d0-4f15-93ff-54132cf5a55e None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:15.815884 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 336/679] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 24 06:22:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:15.816989 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc1015c-213f-4416-9fff-c2d7cd0e3812 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:15.817299 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc1015c-213f-4416-9fff-c2d7cd0e3812 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:15.817664 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc1015c-213f-4416-9fff-c2d7cd0e3812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:15.836966 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-3cc1015c-213f-4416-9fff-c2d7cd0e3812 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:22:15.838495 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-3cc1015c-213f-4416-9fff-c2d7cd0e3812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:22:15.929356 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 337/680] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:22:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:22:15.929356 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc1015c-213f-4416-9fff-c2d7cd0e3812 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:15.946435 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0c13b3-0e7f-4682-a2fa-1b94b69aa991 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:15.998265 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:16.008884 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac5f753-d2dd-4b0d-84c8-ce03bb150307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:16.010348 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac5f753-d2dd-4b0d-84c8-ce03bb150307 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:16.010631 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac5f753-d2dd-4b0d-84c8-ce03bb150307 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:16.011788 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac5f753-d2dd-4b0d-84c8-ce03bb150307 None admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:16.011788 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac5f753-d2dd-4b0d-84c8-ce03bb150307 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:16.011946 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac5f753-d2dd-4b0d-84c8-ce03bb150307 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:16.014590 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac5f753-d2dd-4b0d-84c8-ce03bb150307 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:16.019117 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:16.032991 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 344/681] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 24 06:22:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:16.039559 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac5f753-d2dd-4b0d-84c8-ce03bb150307 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:16.058839 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc1015c-213f-4416-9fff-c2d7cd0e3812 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:16.100869 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4538f0f3-f05f-4bd0-a122-8dc74467e3c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:16.101562 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4538f0f3-f05f-4bd0-a122-8dc74467e3c0 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:16.101840 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4538f0f3-f05f-4bd0-a122-8dc74467e3c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:16.102120 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4538f0f3-f05f-4bd0-a122-8dc74467e3c0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:16.120932 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4538f0f3-f05f-4bd0-a122-8dc74467e3c0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:16.122421 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4538f0f3-f05f-4bd0-a122-8dc74467e3c0 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:16.136403 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-4538f0f3-f05f-4bd0-a122-8dc74467e3c0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:22:16.136403 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:16.136403 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:16.136403 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:16.136403 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:16.136403 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:16.136403 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:16.136403 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:16.136403 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:16.136403 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:16.136403 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:16.136403 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:16.136403 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 06:22:16.136403 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 06:22:16.136403 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 06:22:16.137401 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 06:22:16.137401 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:16.137401 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:16.137401 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:16.137401 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:16.137401 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:16.137401 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:16.137401 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:16.137401 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:16.137401 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:16.137401 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:16.137401 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:16.137401 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:16.137401 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:16.137401 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:16.137401 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:16.137401 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:16.138357 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 06:22:16.138357 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 06:22:16.138357 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:22:16.138357 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:22:16.138357 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:22:16.138357 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:22:16.138357 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:22:16.138357 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:16.144246 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 338/682] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 24 06:22:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:16.144246 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4538f0f3-f05f-4bd0-a122-8dc74467e3c0 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:16.238836 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac5f753-d2dd-4b0d-84c8-ce03bb150307 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:16.291778 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4538f0f3-f05f-4bd0-a122-8dc74467e3c0 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:16.303471 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42818e1-d909-4891-b418-b5c2aab2edd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:16.304649 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42818e1-d909-4891-b418-b5c2aab2edd8 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:16.305004 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42818e1-d909-4891-b418-b5c2aab2edd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:16.305800 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42818e1-d909-4891-b418-b5c2aab2edd8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:16.308761 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:16.322738 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42818e1-d909-4891-b418-b5c2aab2edd8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:16.322738 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42818e1-d909-4891-b418-b5c2aab2edd8 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:16.333135 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc1b959-7310-43b6-8920-f27edc5ea718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:16.333789 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc1b959-7310-43b6-8920-f27edc5ea718 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:16.334072 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc1b959-7310-43b6-8920-f27edc5ea718 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:16.334347 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc1b959-7310-43b6-8920-f27edc5ea718 None admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:16.335053 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc1b959-7310-43b6-8920-f27edc5ea718 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:16.335358 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc1b959-7310-43b6-8920-f27edc5ea718 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:16.335519 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-e42818e1-d909-4891-b418-b5c2aab2edd8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 06:22:16.335519 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:16.335519 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:16.335519 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:16.335519 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:16.335519 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:16.335519 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:16.335519 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:16.335519 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:16.335519 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:16.335519 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:16.335519 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:16.335519 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 06:22:16.335519 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 06:22:16.335519 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 06:22:16.335519 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 06:22:16.336651 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:16.336651 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:16.336651 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 06:22:16.336651 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:22:16.336651 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 06:22:16.336651 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:22:16.336651 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:16.336651 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:16.336651 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:16.336651 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:16.336651 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:16.336651 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:16.336651 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:16.336651 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:16.336651 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:16.336651 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:16.336651 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:16.336651 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:16.338490 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:16.338490 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:16.338490 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 06:22:16.338490 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 06:22:16.338490 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 06:22:16.338490 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 06:22:16.338490 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 06:22:16.338490 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 06:22:16.338490 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 06:22:16.338490 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:16.339753 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc1b959-7310-43b6-8920-f27edc5ea718 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:16.344188 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 345/683] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Aug 24 06:22:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:22:16.344188 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42818e1-d909-4891-b418-b5c2aab2edd8 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:16.357630 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 339/684] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Aug 24 06:22:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:22:16.358852 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc1b959-7310-43b6-8920-f27edc5ea718 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:16.420760 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:16.478067 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc1b959-7310-43b6-8920-f27edc5ea718 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:16.486563 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42818e1-d909-4891-b418-b5c2aab2edd8 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:16.521140 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9d6c35-e9e6-4617-bd96-e3235aab0da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:16.522138 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9d6c35-e9e6-4617-bd96-e3235aab0da8 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:16.522605 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9d6c35-e9e6-4617-bd96-e3235aab0da8 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:16.523276 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9d6c35-e9e6-4617-bd96-e3235aab0da8 None admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:16.524177 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9d6c35-e9e6-4617-bd96-e3235aab0da8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:16.524718 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9d6c35-e9e6-4617-bd96-e3235aab0da8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:16.527053 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9d6c35-e9e6-4617-bd96-e3235aab0da8 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:16.533880 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26793470-0af1-4b20-b60d-846159866d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:16.534518 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26793470-0af1-4b20-b60d-846159866d39 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:16.534801 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26793470-0af1-4b20-b60d-846159866d39 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:16.535080 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26793470-0af1-4b20-b60d-846159866d39 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:16.547439 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26793470-0af1-4b20-b60d-846159866d39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:22:16.553687 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-26793470-0af1-4b20-b60d-846159866d39 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:22:16.553687 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:16.553687 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:16.553687 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:16.553687 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:16.553687 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:16.553687 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:16.553687 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:16.553687 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:16.553687 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:16.553687 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:16.553687 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:16.553687 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:22:16.553687 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:22:16.553687 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:22:16.553687 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:22:16.554851 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:22:16.554851 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:22:16.554851 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:22:16.554851 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:22:16.554851 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:16.554851 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:16.554851 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:16.554851 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:16.554851 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:16.554851 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:16.554851 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:16.554851 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:16.554851 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:16.554851 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:16.554851 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:16.554851 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:16.555783 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:16.555783 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:16.555783 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:16.555783 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:16.555783 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:22:16.555783 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:22:16.555783 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:22:16.555783 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:22:16.555783 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:22:16.555783 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:22:16.555783 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:22:16.555783 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:16.555783 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 346/685] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Aug 24 06:22:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:16.555783 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26793470-0af1-4b20-b60d-846159866d39 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:16.557218 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 340/686] 10.222.0.47 () {56 vars in 1135 bytes} [Wed Aug 24 06:22:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:16.559450 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9d6c35-e9e6-4617-bd96-e3235aab0da8 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:16.594635 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:16.672141 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26793470-0af1-4b20-b60d-846159866d39 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:16.683865 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9d6c35-e9e6-4617-bd96-e3235aab0da8 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:16.692965 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:16.716523 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9709efb3-b45d-418e-9107-2105b2cea80d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:16.717145 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9709efb3-b45d-418e-9107-2105b2cea80d None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:16.717455 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9709efb3-b45d-418e-9107-2105b2cea80d None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:16.717874 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9709efb3-b45d-418e-9107-2105b2cea80d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:16.718604 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9709efb3-b45d-418e-9107-2105b2cea80d None admin] RBAC: Adding query filter params (role.id=1712d85aedaf4189bf4bc89094d888c3, scope.project.id=c02e49a89cdd40d885bfa816f06c58a1, user.id=f0ba3f03ec15475ab6ebf64994ad742c) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:16.718915 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9709efb3-b45d-418e-9107-2105b2cea80d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:16.720388 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9709efb3-b45d-418e-9107-2105b2cea80d None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:16.726485 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6990b7-d7ce-4be5-95cd-847f0a0922b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:16.726981 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 347/687] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 24 06:22:16 2022] GET /identity/v3/role_assignments?user.id=f0ba3f03ec15475ab6ebf64994ad742c&scope.project.id=c02e49a89cdd40d885bfa816f06c58a1&role.id=1712d85aedaf4189bf4bc89094d888c3 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:16.727613 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6990b7-d7ce-4be5-95cd-847f0a0922b7 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:16.727856 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9709efb3-b45d-418e-9107-2105b2cea80d None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:16.728346 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6990b7-d7ce-4be5-95cd-847f0a0922b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:16.728652 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6990b7-d7ce-4be5-95cd-847f0a0922b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:16.842319 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6990b7-d7ce-4be5-95cd-847f0a0922b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:16.844237 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6990b7-d7ce-4be5-95cd-847f0a0922b7 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:16.870738 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9709efb3-b45d-418e-9107-2105b2cea80d None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:16.883423 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:16.917483 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732a73ec-6fc4-4bc9-8455-c9ad52e6ef3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:16.918808 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732a73ec-6fc4-4bc9-8455-c9ad52e6ef3e None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:16.919505 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732a73ec-6fc4-4bc9-8455-c9ad52e6ef3e None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:16.919992 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732a73ec-6fc4-4bc9-8455-c9ad52e6ef3e None admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:16.921019 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732a73ec-6fc4-4bc9-8455-c9ad52e6ef3e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:16.921843 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732a73ec-6fc4-4bc9-8455-c9ad52e6ef3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:16.923381 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732a73ec-6fc4-4bc9-8455-c9ad52e6ef3e None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:16.935376 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 348/688] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 24 06:22:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:16.937168 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c921d004-5ef8-4c55-958d-47c366a97902 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:16.937168 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c921d004-5ef8-4c55-958d-47c366a97902 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:16.937394 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c921d004-5ef8-4c55-958d-47c366a97902 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:16.939162 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 349/689] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:22:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:22:16.941705 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c921d004-5ef8-4c55-958d-47c366a97902 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:16.987930 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 341/690] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 24 06:22:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 06:22:16.989945 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe48e41-9197-4f40-9256-5a8f82658933 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:16.990410 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe48e41-9197-4f40-9256-5a8f82658933 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:16.990856 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe48e41-9197-4f40-9256-5a8f82658933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:17.003071 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:17.021358 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-bbe48e41-9197-4f40-9256-5a8f82658933 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:22:17.023423 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-bbe48e41-9197-4f40-9256-5a8f82658933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:22:17.072335 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c921d004-5ef8-4c55-958d-47c366a97902 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:17.111512 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6697371a-d67f-478b-8439-a9f10a13a6ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:17.112803 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6697371a-d67f-478b-8439-a9f10a13a6ac None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:17.113097 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6697371a-d67f-478b-8439-a9f10a13a6ac None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:17.113402 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6697371a-d67f-478b-8439-a9f10a13a6ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:17.125369 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 342/691] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:22:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:22:17.127622 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe48e41-9197-4f40-9256-5a8f82658933 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:17.131508 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6697371a-d67f-478b-8439-a9f10a13a6ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:17.133006 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6697371a-d67f-478b-8439-a9f10a13a6ac None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:17.169527 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-6697371a-d67f-478b-8439-a9f10a13a6ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:17.169527 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:17.169527 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:17.169527 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:17.169527 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:17.169527 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:17.169527 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:17.169527 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:17.169527 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:17.169527 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:17.169527 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:17.169527 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:17.169527 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:22:17.169527 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:22:17.169527 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:22:17.169527 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:22:17.170530 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:17.170530 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:17.170530 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:17.170530 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:17.170530 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:17.170530 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:17.170530 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:17.170530 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:17.170530 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:17.170530 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:17.170530 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:17.170530 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:17.170530 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:17.170530 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:17.170530 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:17.170530 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:17.170530 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:22:17.171419 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:22:17.171419 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:17.171419 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:17.177222 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 350/692] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:22:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:17.179644 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6697371a-d67f-478b-8439-a9f10a13a6ac None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:17.221655 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:17.256345 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe48e41-9197-4f40-9256-5a8f82658933 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:17.294966 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc826eb-0809-46df-943b-894226908d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:17.295618 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc826eb-0809-46df-943b-894226908d57 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:17.295899 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc826eb-0809-46df-943b-894226908d57 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:17.296180 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc826eb-0809-46df-943b-894226908d57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:17.308159 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc826eb-0809-46df-943b-894226908d57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:17.309848 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc826eb-0809-46df-943b-894226908d57 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:17.327258 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-ecc826eb-0809-46df-943b-894226908d57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:17.327258 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:17.327258 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:17.327258 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:17.327258 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:17.327258 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:17.327258 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:17.327258 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:17.327258 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:17.327258 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:17.327258 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:17.327258 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:17.327258 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:22:17.327258 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:22:17.327258 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:22:17.327258 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:22:17.329594 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:17.329594 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:17.329594 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:17.329594 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:17.329594 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:17.329594 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:17.329594 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:17.329594 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:17.329594 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:17.329594 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:17.329594 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:17.329594 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:17.329594 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:17.329594 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:17.329594 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:17.329594 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:17.329594 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:22:17.331687 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:22:17.331687 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:17.331687 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:17.331687 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 343/693] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:22:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:17.343268 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc826eb-0809-46df-943b-894226908d57 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:17.365901 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6697371a-d67f-478b-8439-a9f10a13a6ac None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:17.370260 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:17.438164 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6813a54-9381-4020-8833-288a7c67641f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:17.438934 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6813a54-9381-4020-8833-288a7c67641f None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:17.439096 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6813a54-9381-4020-8833-288a7c67641f None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:17.439314 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6813a54-9381-4020-8833-288a7c67641f None admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:17.440019 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6813a54-9381-4020-8833-288a7c67641f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:17.440360 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6813a54-9381-4020-8833-288a7c67641f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:17.441857 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6813a54-9381-4020-8833-288a7c67641f None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:17.466351 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 351/694] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 24 06:22:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:17.474885 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6813a54-9381-4020-8833-288a7c67641f None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:17.499257 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc826eb-0809-46df-943b-894226908d57 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:17.524458 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:17.566720 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ef6325-59e3-40c0-962b-ec420b7cd848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:17.567384 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ef6325-59e3-40c0-962b-ec420b7cd848 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:17.567611 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ef6325-59e3-40c0-962b-ec420b7cd848 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:17.567875 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ef6325-59e3-40c0-962b-ec420b7cd848 None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:17.568528 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ef6325-59e3-40c0-962b-ec420b7cd848 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:17.568830 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ef6325-59e3-40c0-962b-ec420b7cd848 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:17.570217 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ef6325-59e3-40c0-962b-ec420b7cd848 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:17.592613 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 344/695] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 24 06:22:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:22:17.593807 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ef6325-59e3-40c0-962b-ec420b7cd848 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:17.651380 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6813a54-9381-4020-8833-288a7c67641f None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:17.690700 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e50bf1c-ac34-4717-9203-a88a93d07ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:17.691268 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e50bf1c-ac34-4717-9203-a88a93d07ff9 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:17.691551 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e50bf1c-ac34-4717-9203-a88a93d07ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:17.691830 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e50bf1c-ac34-4717-9203-a88a93d07ff9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:17.701751 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e50bf1c-ac34-4717-9203-a88a93d07ff9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:17.703358 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e50bf1c-ac34-4717-9203-a88a93d07ff9 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:17.706634 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ef6325-59e3-40c0-962b-ec420b7cd848 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:17.716847 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-9e50bf1c-ac34-4717-9203-a88a93d07ff9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 06:22:17.716847 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:17.716847 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:17.716847 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:17.716847 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:17.716847 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:17.716847 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:17.716847 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:17.716847 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:17.716847 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:17.716847 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:17.716847 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:17.716847 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 06:22:17.716847 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 06:22:17.716847 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 06:22:17.716847 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 06:22:17.717886 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:17.717886 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:17.717886 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 06:22:17.717886 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:22:17.717886 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 06:22:17.717886 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:22:17.717886 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:17.717886 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:17.717886 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:17.717886 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:17.717886 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:17.717886 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:17.717886 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:17.717886 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:17.717886 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:17.717886 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:17.717886 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:17.717886 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:17.720347 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:17.720347 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:17.720347 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 06:22:17.720347 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 06:22:17.720347 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 06:22:17.720347 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 06:22:17.720347 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 06:22:17.720347 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 06:22:17.720347 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 06:22:17.720347 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:17.720347 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 352/696] 10.222.0.47 () {56 vars in 1093 bytes} [Wed Aug 24 06:22:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:17.720347 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e50bf1c-ac34-4717-9203-a88a93d07ff9 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:17.740121 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:17.756300 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502cb45e-f4a4-42c0-9603-3fb69cfc5bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:17.756959 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502cb45e-f4a4-42c0-9603-3fb69cfc5bce None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:17.757271 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502cb45e-f4a4-42c0-9603-3fb69cfc5bce None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:17.758344 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502cb45e-f4a4-42c0-9603-3fb69cfc5bce None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:17.758344 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502cb45e-f4a4-42c0-9603-3fb69cfc5bce None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:17.760842 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502cb45e-f4a4-42c0-9603-3fb69cfc5bce None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:17.781509 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-502cb45e-f4a4-42c0-9603-3fb69cfc5bce None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 06:22:17.781509 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:17.781509 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 06:22:17.781509 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 06:22:17.781509 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 06:22:17.781509 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 06:22:17.781509 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 06:22:17.781509 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:22:17.781509 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 06:22:17.781509 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:17.781509 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 06:22:17.781509 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:17.781509 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:17.781509 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:17.781509 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:17.781509 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:17.781509 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:17.783068 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:17.783068 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:17.783068 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:17.783068 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:17.783068 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:17.783068 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:17.783068 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 06:22:17.783068 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 06:22:17.783068 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 06:22:17.783068 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 06:22:17.783068 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:17.783068 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:17.783068 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:17.783068 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:17.783068 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:17.783068 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:17.784130 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:17.784130 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:17.784130 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:17.784130 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:17.784130 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:17.784130 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:17.784130 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:17.784130 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:17.784130 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:17.784130 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:17.784130 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 06:22:17.784130 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:22:17.784130 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 06:22:17.784130 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:17.789798 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 345/697] 10.222.0.47 () {56 vars in 1129 bytes} [Wed Aug 24 06:22:17 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:17.789798 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502cb45e-f4a4-42c0-9603-3fb69cfc5bce None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:17.833568 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:17.850878 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e50bf1c-ac34-4717-9203-a88a93d07ff9 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:17.890139 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd296ef-aedb-4351-93e6-72d31959e8e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:17.891220 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd296ef-aedb-4351-93e6-72d31959e8e5 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:17.891717 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd296ef-aedb-4351-93e6-72d31959e8e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:17.892186 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd296ef-aedb-4351-93e6-72d31959e8e5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:17.900783 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd296ef-aedb-4351-93e6-72d31959e8e5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:17.900970 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502cb45e-f4a4-42c0-9603-3fb69cfc5bce None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:17.902780 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd296ef-aedb-4351-93e6-72d31959e8e5 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:17.915546 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-2bd296ef-aedb-4351-93e6-72d31959e8e5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:22:17.915546 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:17.915546 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:17.915546 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:17.915546 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:17.915546 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:17.915546 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:17.915546 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:17.915546 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:17.915546 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:17.915546 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:17.915546 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:17.915546 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 06:22:17.915546 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 06:22:17.915546 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 06:22:17.916582 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 06:22:17.916582 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:17.916582 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:17.916582 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:17.916582 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:17.916582 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:17.916582 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:17.916582 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:17.916582 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:17.916582 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:17.916582 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:17.916582 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:17.916582 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:17.916582 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:17.916582 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:17.916582 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:17.916582 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:17.917513 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 06:22:17.917513 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 06:22:17.917513 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:22:17.917513 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:22:17.917513 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:22:17.917513 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:22:17.917513 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:22:17.917513 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:17.924597 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 353/698] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 24 06:22:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:17.926066 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd296ef-aedb-4351-93e6-72d31959e8e5 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:17.939747 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c7ee14-0864-4be5-a95d-891f1214013b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:17.940419 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c7ee14-0864-4be5-a95d-891f1214013b None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:17.940650 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c7ee14-0864-4be5-a95d-891f1214013b None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:17.941002 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c7ee14-0864-4be5-a95d-891f1214013b None admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:17.941720 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c7ee14-0864-4be5-a95d-891f1214013b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:17.942031 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c7ee14-0864-4be5-a95d-891f1214013b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:17.943606 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c7ee14-0864-4be5-a95d-891f1214013b None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:17.969588 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 346/699] 10.222.0.47 () {56 vars in 1133 bytes} [Wed Aug 24 06:22:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:17.970559 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c7ee14-0864-4be5-a95d-891f1214013b None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:18.011051 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:18.042663 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd296ef-aedb-4351-93e6-72d31959e8e5 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:18.079972 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:18.083392 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9140c52-8627-4e28-a014-94fdc8f287fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:18.084015 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9140c52-8627-4e28-a014-94fdc8f287fb None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:18.084297 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9140c52-8627-4e28-a014-94fdc8f287fb None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:18.084571 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9140c52-8627-4e28-a014-94fdc8f287fb None admin] PATH_INFO: `/v3/services` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:18.085230 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9140c52-8627-4e28-a014-94fdc8f287fb None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:18.085539 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9140c52-8627-4e28-a014-94fdc8f287fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:18.086925 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9140c52-8627-4e28-a014-94fdc8f287fb None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:18.087719 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c7ee14-0864-4be5-a95d-891f1214013b None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:18.101161 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 354/700] 10.222.0.47 () {56 vars in 1124 bytes} [Wed Aug 24 06:22:17 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:18.106267 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9140c52-8627-4e28-a014-94fdc8f287fb None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:18.131916 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83860db-538a-4ee3-9ab7-f8478c7072d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:18.132534 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83860db-538a-4ee3-9ab7-f8478c7072d4 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:18.132814 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83860db-538a-4ee3-9ab7-f8478c7072d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:18.133092 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83860db-538a-4ee3-9ab7-f8478c7072d4 None admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:18.133822 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83860db-538a-4ee3-9ab7-f8478c7072d4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:18.134132 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83860db-538a-4ee3-9ab7-f8478c7072d4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:18.135846 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83860db-538a-4ee3-9ab7-f8478c7072d4 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:18.153076 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 347/701] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Aug 24 06:22:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:18.154257 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc57647-0b5b-4fe6-81c5-0f65b86491d7 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:18.154544 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc57647-0b5b-4fe6-81c5-0f65b86491d7 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:18.154821 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc57647-0b5b-4fe6-81c5-0f65b86491d7 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:18.160429 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 348/702] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:22:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:22:18.160429 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc57647-0b5b-4fe6-81c5-0f65b86491d7 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:18.232061 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9140c52-8627-4e28-a014-94fdc8f287fb None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:18.274344 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881a2aff-85ec-4700-856f-343b6e5d6d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:18.276050 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881a2aff-85ec-4700-856f-343b6e5d6d4c None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:18.276599 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881a2aff-85ec-4700-856f-343b6e5d6d4c None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:18.277054 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881a2aff-85ec-4700-856f-343b6e5d6d4c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:18.277484 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc57647-0b5b-4fe6-81c5-0f65b86491d7 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:18.291917 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:18.316874 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96599a9-94dc-4688-a554-29e6e6284e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:18.322595 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96599a9-94dc-4688-a554-29e6e6284e1a None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:18.322595 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96599a9-94dc-4688-a554-29e6e6284e1a None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:18.322595 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96599a9-94dc-4688-a554-29e6e6284e1a None admin] PATH_INFO: `/v3/services` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:18.322595 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96599a9-94dc-4688-a554-29e6e6284e1a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:18.322595 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96599a9-94dc-4688-a554-29e6e6284e1a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:18.322595 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96599a9-94dc-4688-a554-29e6e6284e1a None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:18.337223 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 349/703] 10.222.0.47 () {56 vars in 1124 bytes} [Wed Aug 24 06:22:18 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:18.337223 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96599a9-94dc-4688-a554-29e6e6284e1a None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:18.363207 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:18.385815 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881a2aff-85ec-4700-856f-343b6e5d6d4c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:18.387323 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881a2aff-85ec-4700-856f-343b6e5d6d4c None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:18.484923 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96599a9-94dc-4688-a554-29e6e6284e1a None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:18.533077 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404aa1fc-2012-41fd-a4c3-05cfcdd30bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:18.533852 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404aa1fc-2012-41fd-a4c3-05cfcdd30bab None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:18.538953 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404aa1fc-2012-41fd-a4c3-05cfcdd30bab None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:18.538953 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404aa1fc-2012-41fd-a4c3-05cfcdd30bab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:18.538953 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404aa1fc-2012-41fd-a4c3-05cfcdd30bab None admin] RBAC: Adding query filter params (role.id=1712d85aedaf4189bf4bc89094d888c3, scope.project.id=c02e49a89cdd40d885bfa816f06c58a1, user.id=02f1cbaf2cd6427585f298c9d58c319a) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:18.538953 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404aa1fc-2012-41fd-a4c3-05cfcdd30bab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:18.538953 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404aa1fc-2012-41fd-a4c3-05cfcdd30bab None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:18.544500 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 350/704] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 24 06:22:18 2022] GET /identity/v3/role_assignments?user.id=02f1cbaf2cd6427585f298c9d58c319a&scope.project.id=c02e49a89cdd40d885bfa816f06c58a1&role.id=1712d85aedaf4189bf4bc89094d888c3 => generated 608 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:18.546015 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9df5db-f859-4cae-838b-d54029398f31 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:18.546369 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9df5db-f859-4cae-838b-d54029398f31 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:18.546717 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9df5db-f859-4cae-838b-d54029398f31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:18.548744 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 355/705] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 24 06:22:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 06:22:18.560972 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881a2aff-85ec-4700-856f-343b6e5d6d4c None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:18.566876 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-2e9df5db-f859-4cae-838b-d54029398f31 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:22:18.568246 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-2e9df5db-f859-4cae-838b-d54029398f31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:22:18.669567 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:18.699975 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 351/706] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:22:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:22:18.768170 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881a2aff-85ec-4700-856f-343b6e5d6d4c None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:18.810853 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7d2f2f-5adc-4633-98f4-72fe797cf8f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:18.811505 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7d2f2f-5adc-4633-98f4-72fe797cf8f1 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:18.811789 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7d2f2f-5adc-4633-98f4-72fe797cf8f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:18.812063 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7d2f2f-5adc-4633-98f4-72fe797cf8f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:18.824548 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7d2f2f-5adc-4633-98f4-72fe797cf8f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:18.826749 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7d2f2f-5adc-4633-98f4-72fe797cf8f1 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:18.844687 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-3d7d2f2f-5adc-4633-98f4-72fe797cf8f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:18.844687 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:18.844687 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:18.844687 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:18.844687 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:18.844687 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:18.844687 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:18.844687 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:18.844687 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:18.844687 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:18.844687 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:18.844687 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:18.844687 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:22:18.844687 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:22:18.844687 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:22:18.844687 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:22:18.845732 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:18.845732 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:18.845732 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:18.845732 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:18.845732 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:18.845732 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:18.845732 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:18.845732 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:18.845732 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:18.845732 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:18.845732 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:18.845732 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:18.845732 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:18.845732 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:18.845732 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:18.845732 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:18.845732 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:22:18.846639 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:22:18.846639 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:18.846639 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:18.846639 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 356/707] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:22:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:22:18.852268 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9df5db-f859-4cae-838b-d54029398f31 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:18.870956 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:18.952945 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ddeb5d-c4ea-4f82-a0ac-ad6d924b4d01 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:18.953379 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ddeb5d-c4ea-4f82-a0ac-ad6d924b4d01 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:18.953568 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ddeb5d-c4ea-4f82-a0ac-ad6d924b4d01 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:18.954867 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 357/708] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:22:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:22:18.960375 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32db7a6-9451-460f-9a37-89bc39e3e299 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:18.960658 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32db7a6-9451-460f-9a37-89bc39e3e299 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:18.960927 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32db7a6-9451-460f-9a37-89bc39e3e299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:18.981043 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9df5db-f859-4cae-838b-d54029398f31 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:18.990384 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-f32db7a6-9451-460f-9a37-89bc39e3e299 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:22:18.991647 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-f32db7a6-9451-460f-9a37-89bc39e3e299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:22:19.037599 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68349cc4-2b6b-467b-ac47-90df87f118e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:19.038179 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68349cc4-2b6b-467b-ac47-90df87f118e4 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:19.038462 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68349cc4-2b6b-467b-ac47-90df87f118e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:19.038739 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68349cc4-2b6b-467b-ac47-90df87f118e4 None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:19.039397 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68349cc4-2b6b-467b-ac47-90df87f118e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:19.039709 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68349cc4-2b6b-467b-ac47-90df87f118e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:19.041713 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68349cc4-2b6b-467b-ac47-90df87f118e4 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:19.057641 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 352/709] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 24 06:22:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:19.063132 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68349cc4-2b6b-467b-ac47-90df87f118e4 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:19.095978 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 358/710] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:22:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:22:19.096863 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32db7a6-9451-460f-9a37-89bc39e3e299 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:19.122679 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:19.156237 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:19.181957 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68349cc4-2b6b-467b-ac47-90df87f118e4 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:19.208715 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32db7a6-9451-460f-9a37-89bc39e3e299 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:19.249831 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f7eb1b-d44a-4006-805b-d97186b15257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:19.250477 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f7eb1b-d44a-4006-805b-d97186b15257 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:19.250800 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f7eb1b-d44a-4006-805b-d97186b15257 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:19.251076 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f7eb1b-d44a-4006-805b-d97186b15257 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:19.263649 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f7eb1b-d44a-4006-805b-d97186b15257 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:19.265212 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f7eb1b-d44a-4006-805b-d97186b15257 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:19.278841 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884ec8cb-158f-478d-9ed9-563aa8ee0e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:19.279491 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884ec8cb-158f-478d-9ed9-563aa8ee0e8e None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:19.279774 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884ec8cb-158f-478d-9ed9-563aa8ee0e8e None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:19.280043 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884ec8cb-158f-478d-9ed9-563aa8ee0e8e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:19.283073 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-f4f7eb1b-d44a-4006-805b-d97186b15257 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:19.283073 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:19.283073 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:19.283073 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:19.283073 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:19.283073 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:19.283073 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:19.283073 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:19.283073 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:19.283073 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:19.283073 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:19.283073 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:19.283073 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:22:19.283073 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:22:19.283073 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:22:19.283073 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:22:19.285132 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:19.285132 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:19.285132 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:19.285132 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:19.285132 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:19.285132 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:19.285132 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:19.285132 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:19.285132 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:19.285132 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:19.285132 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:19.285132 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:19.285132 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:19.285132 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:19.285132 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:19.285132 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:19.285132 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:22:19.287165 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:22:19.287165 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:19.287165 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:19.287165 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 359/711] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:22:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:19.294604 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f7eb1b-d44a-4006-805b-d97186b15257 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:19.299960 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884ec8cb-158f-478d-9ed9-563aa8ee0e8e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:19.301480 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884ec8cb-158f-478d-9ed9-563aa8ee0e8e None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:19.324210 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-884ec8cb-158f-478d-9ed9-563aa8ee0e8e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:22:19.324210 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:19.324210 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:19.324210 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:19.324210 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:19.324210 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:19.324210 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:19.324210 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:19.324210 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:19.324210 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:19.324210 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:19.324210 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:19.324210 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 06:22:19.324210 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 06:22:19.324210 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 06:22:19.326446 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 06:22:19.326446 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:19.326446 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:19.326446 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:19.326446 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:19.326446 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:19.326446 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:19.326446 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:19.326446 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:19.326446 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:19.326446 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:19.326446 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:19.326446 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:19.326446 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:19.326446 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:19.326446 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:19.326446 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:19.328084 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 06:22:19.328084 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 06:22:19.328084 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:22:19.328084 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:22:19.328084 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:22:19.328084 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:22:19.328084 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:22:19.328084 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:19.328084 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 353/712] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 24 06:22:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:22:19.333064 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884ec8cb-158f-478d-9ed9-563aa8ee0e8e None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:19.454806 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:19.491205 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f7eb1b-d44a-4006-805b-d97186b15257 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:19.492534 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884ec8cb-158f-478d-9ed9-563aa8ee0e8e None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:19.527898 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:19.533673 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a8b928-5746-459a-908f-b3a6ebba785f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:19.534584 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a8b928-5746-459a-908f-b3a6ebba785f None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:19.534957 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a8b928-5746-459a-908f-b3a6ebba785f None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:19.535324 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a8b928-5746-459a-908f-b3a6ebba785f None admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:19.536185 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a8b928-5746-459a-908f-b3a6ebba785f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:19.536808 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a8b928-5746-459a-908f-b3a6ebba785f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:19.538494 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a8b928-5746-459a-908f-b3a6ebba785f None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:19.539451 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ab3465-038d-4b7e-b7e6-37e9f9bc5d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:19.544071 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ab3465-038d-4b7e-b7e6-37e9f9bc5d03 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:19.544071 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ab3465-038d-4b7e-b7e6-37e9f9bc5d03 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:19.544071 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ab3465-038d-4b7e-b7e6-37e9f9bc5d03 None admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:19.544071 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ab3465-038d-4b7e-b7e6-37e9f9bc5d03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:19.544071 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ab3465-038d-4b7e-b7e6-37e9f9bc5d03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:19.544071 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ab3465-038d-4b7e-b7e6-37e9f9bc5d03 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:19.556056 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 354/713] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Aug 24 06:22:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:22:19.562027 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 360/714] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 24 06:22:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:22:19.562522 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a8b928-5746-459a-908f-b3a6ebba785f None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:19.563398 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ab3465-038d-4b7e-b7e6-37e9f9bc5d03 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:19.672716 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a8b928-5746-459a-908f-b3a6ebba785f None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:19.679338 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ab3465-038d-4b7e-b7e6-37e9f9bc5d03 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:19.712347 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbafa0d7-b684-4514-9b77-06ad85293a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:19.713794 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbafa0d7-b684-4514-9b77-06ad85293a2a None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:19.714311 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbafa0d7-b684-4514-9b77-06ad85293a2a None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:19.714787 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbafa0d7-b684-4514-9b77-06ad85293a2a None admin] PATH_INFO: `/v3/services/network` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:19.715021 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:19.715816 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbafa0d7-b684-4514-9b77-06ad85293a2a None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:19.717606 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbafa0d7-b684-4514-9b77-06ad85293a2a None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:19.720861 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327b0f61-d74b-4dc3-9f42-17f1d4cd3cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:19.721708 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327b0f61-d74b-4dc3-9f42-17f1d4cd3cf0 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:19.721996 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327b0f61-d74b-4dc3-9f42-17f1d4cd3cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:19.727094 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327b0f61-d74b-4dc3-9f42-17f1d4cd3cf0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:19.727094 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327b0f61-d74b-4dc3-9f42-17f1d4cd3cf0 None admin] RBAC: Adding query filter params (role.id=486144ad1bcd417e86acdbc1d14327bf, scope.project.id=c02e49a89cdd40d885bfa816f06c58a1, user.id=1f4f49c1b08c460f8ba6c451c9c92c2a) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:19.727094 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327b0f61-d74b-4dc3-9f42-17f1d4cd3cf0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:19.727094 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327b0f61-d74b-4dc3-9f42-17f1d4cd3cf0 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:19.729741 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-cbafa0d7-b684-4514-9b77-06ad85293a2a None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 06:22:19.729741 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:19.729741 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 06:22:19.729741 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 06:22:19.729741 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 06:22:19.729741 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 06:22:19.729741 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 06:22:19.729741 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:22:19.729741 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 06:22:19.729741 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:19.729741 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 06:22:19.729741 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:19.729741 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:19.729741 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:19.729741 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:19.729741 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:19.729741 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:19.730762 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:19.730762 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:19.730762 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:19.730762 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:19.730762 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:19.730762 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:19.730762 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 06:22:19.730762 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 06:22:19.730762 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 06:22:19.730762 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 06:22:19.730762 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:19.730762 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:19.730762 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:19.730762 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:19.730762 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:19.730762 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:19.731647 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:19.731647 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:19.731647 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:19.731647 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:19.731647 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:19.731647 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:19.731647 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:19.731647 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:19.731647 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:19.731647 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:19.731647 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 06:22:19.731647 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:22:19.731647 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 06:22:19.731647 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:19.732389 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 361/715] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 24 06:22:19 2022] GET /identity/v3/role_assignments?user.id=1f4f49c1b08c460f8ba6c451c9c92c2a&scope.project.id=c02e49a89cdd40d885bfa816f06c58a1&role.id=486144ad1bcd417e86acdbc1d14327bf => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:22:19.733142 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327b0f61-d74b-4dc3-9f42-17f1d4cd3cf0 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:19.739672 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 355/716] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 24 06:22:19 2022] GET /identity/v3/services/network => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:19.740987 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbafa0d7-b684-4514-9b77-06ad85293a2a None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:19.814479 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:19.870919 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbafa0d7-b684-4514-9b77-06ad85293a2a None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:19.874439 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327b0f61-d74b-4dc3-9f42-17f1d4cd3cf0 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:19.910855 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41d35db-a00a-48b7-b467-d91711ae1df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:19.911940 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41d35db-a00a-48b7-b467-d91711ae1df7 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:19.912518 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41d35db-a00a-48b7-b467-d91711ae1df7 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:19.913068 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41d35db-a00a-48b7-b467-d91711ae1df7 None admin] PATH_INFO: `/v3/services` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:19.914188 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41d35db-a00a-48b7-b467-d91711ae1df7 None admin] RBAC: Adding query filter params (name=network) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:19.914791 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41d35db-a00a-48b7-b467-d91711ae1df7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:19.915450 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f300e9-84a5-4eca-a1d5-a34b32282970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:19.916107 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f300e9-84a5-4eca-a1d5-a34b32282970 None admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:19.916439 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f300e9-84a5-4eca-a1d5-a34b32282970 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:19.916722 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f300e9-84a5-4eca-a1d5-a34b32282970 None admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:19.917415 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f300e9-84a5-4eca-a1d5-a34b32282970 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:19.917882 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41d35db-a00a-48b7-b467-d91711ae1df7 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:19.918867 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f300e9-84a5-4eca-a1d5-a34b32282970 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:19.923776 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 356/717] 10.222.0.47 () {56 vars in 1112 bytes} [Wed Aug 24 06:22:19 2022] GET /identity/v3/services?name=network => generated 124 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:19.929848 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41d35db-a00a-48b7-b467-d91711ae1df7 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:19.997121 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 362/718] 10.222.0.47 () {60 vars in 1132 bytes} [Wed Aug 24 06:22:19 2022] POST /identity/v3/users => generated 254 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:22:20.055052 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41d35db-a00a-48b7-b467-d91711ae1df7 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:20.080930 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:20.106763 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b40b03-ee0b-4ec3-bdb0-8c4143b47528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:20.107785 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b40b03-ee0b-4ec3-bdb0-8c4143b47528 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:20.108289 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b40b03-ee0b-4ec3-bdb0-8c4143b47528 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:20.108773 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b40b03-ee0b-4ec3-bdb0-8c4143b47528 None admin] PATH_INFO: `/v3/services` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:20.109668 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b40b03-ee0b-4ec3-bdb0-8c4143b47528 None admin] RBAC: Adding query filter params (type=network) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:20.110209 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b40b03-ee0b-4ec3-bdb0-8c4143b47528 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:20.111836 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b40b03-ee0b-4ec3-bdb0-8c4143b47528 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:20.117728 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 357/719] 10.222.0.47 () {56 vars in 1112 bytes} [Wed Aug 24 06:22:19 2022] GET /identity/v3/services?type=network => generated 124 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:20.756383 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95058e4-a4a3-4588-b3bd-aa7d7e171443 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:20.756989 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95058e4-a4a3-4588-b3bd-aa7d7e171443 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:20.756989 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95058e4-a4a3-4588-b3bd-aa7d7e171443 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:20.758646 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 363/720] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:22:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:22:20.765215 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f334120-5e6f-4961-bbc1-e0c513b0ad53 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:20.765701 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f334120-5e6f-4961-bbc1-e0c513b0ad53 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:20.765777 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f334120-5e6f-4961-bbc1-e0c513b0ad53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:20.790897 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-2f334120-5e6f-4961-bbc1-e0c513b0ad53 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:22:20.792186 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-2f334120-5e6f-4961-bbc1-e0c513b0ad53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:22:20.900182 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 358/721] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:22:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:22:21.192375 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a76a6da-9148-45ff-89e3-3a698494543d None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:21.192564 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a76a6da-9148-45ff-89e3-3a698494543d None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:21.192846 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a76a6da-9148-45ff-89e3-3a698494543d None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:21.194801 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 364/722] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:22:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:22:21.206619 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc90d1b2-4fb8-46a9-bead-276cd92dfa85 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:21.207174 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc90d1b2-4fb8-46a9-bead-276cd92dfa85 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:21.207699 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc90d1b2-4fb8-46a9-bead-276cd92dfa85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:21.234365 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-dc90d1b2-4fb8-46a9-bead-276cd92dfa85 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:22:21.235921 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-dc90d1b2-4fb8-46a9-bead-276cd92dfa85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:22:21.318429 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 359/723] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:22:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:22:21.382883 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a76a6da-9148-45ff-89e3-3a698494543d None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:21.472384 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:21.555695 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a76a6da-9148-45ff-89e3-3a698494543d None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:21.605261 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f6b4b4-c68d-494f-9c9d-2a9fa66b31c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:21.605882 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f6b4b4-c68d-494f-9c9d-2a9fa66b31c7 None admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:21.606170 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f6b4b4-c68d-494f-9c9d-2a9fa66b31c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:21.606442 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f6b4b4-c68d-494f-9c9d-2a9fa66b31c7 None admin] PATH_INFO: `/v3/services` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:21.607093 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f6b4b4-c68d-494f-9c9d-2a9fa66b31c7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:21.608467 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f6b4b4-c68d-494f-9c9d-2a9fa66b31c7 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:21.643879 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc90d1b2-4fb8-46a9-bead-276cd92dfa85 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:21.699118 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:21.721392 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 365/724] 10.222.0.47 () {60 vars in 1141 bytes} [Wed Aug 24 06:22:21 2022] POST /identity/v3/services => generated 251 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:22:21.774519 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc90d1b2-4fb8-46a9-bead-276cd92dfa85 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:21.846726 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a8241d-0e0b-46c1-b2d1-f7244774d994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:21.847403 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a8241d-0e0b-46c1-b2d1-f7244774d994 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:21.847687 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a8241d-0e0b-46c1-b2d1-f7244774d994 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:21.847960 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a8241d-0e0b-46c1-b2d1-f7244774d994 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:21.867514 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a8241d-0e0b-46c1-b2d1-f7244774d994 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:22:21.873242 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-50a8241d-0e0b-46c1-b2d1-f7244774d994 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:22:21.873242 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:21.873242 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:21.873242 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:21.873242 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:21.873242 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:21.873242 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:21.873242 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:21.873242 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:21.873242 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:21.873242 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:21.873242 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:21.873242 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:22:21.873242 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:22:21.873242 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:22:21.873242 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:22:21.874264 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:22:21.874264 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:22:21.874264 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:22:21.874264 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:22:21.874264 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:21.874264 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:21.874264 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:21.874264 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:21.874264 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:21.874264 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:21.874264 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:21.874264 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:21.874264 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:21.874264 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:21.874264 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:21.874264 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:21.875544 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:21.875544 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:21.875544 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:21.875544 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:21.875544 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:22:21.875544 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:22:21.875544 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:22:21.875544 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:22:21.875544 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:22:21.875544 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:22:21.875544 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:22:21.875544 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:21.875544 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 360/725] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Aug 24 06:22:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:21.882855 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f6b4b4-c68d-494f-9c9d-2a9fa66b31c7 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:22.032721 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f6b4b4-c68d-494f-9c9d-2a9fa66b31c7 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:22.048258 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:22.077707 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea989b9b-b64c-49f6-8451-8ac13a14df46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:22.079872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea989b9b-b64c-49f6-8451-8ac13a14df46 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:22.080565 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea989b9b-b64c-49f6-8451-8ac13a14df46 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:22.080995 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea989b9b-b64c-49f6-8451-8ac13a14df46 None admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:22.082072 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea989b9b-b64c-49f6-8451-8ac13a14df46 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:22.082545 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea989b9b-b64c-49f6-8451-8ac13a14df46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:22.085106 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea989b9b-b64c-49f6-8451-8ac13a14df46 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:22.101551 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 366/726] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 24 06:22:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:22.108420 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a8241d-0e0b-46c1-b2d1-f7244774d994 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:22.155907 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490e1d8f-33d1-49a0-b3f8-d7f3b33a6d79 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:22.156300 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490e1d8f-33d1-49a0-b3f8-d7f3b33a6d79 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:22.156577 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490e1d8f-33d1-49a0-b3f8-d7f3b33a6d79 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:22.159236 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 367/727] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:22:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 06:22:22.168305 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df64c3e-f51a-43db-b9bb-df673944bed6 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:22.168620 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df64c3e-f51a-43db-b9bb-df673944bed6 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:22.168862 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df64c3e-f51a-43db-b9bb-df673944bed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:22.203899 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-7df64c3e-f51a-43db-b9bb-df673944bed6 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:22:22.205246 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-7df64c3e-f51a-43db-b9bb-df673944bed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:22:22.227650 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:22.232152 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a8241d-0e0b-46c1-b2d1-f7244774d994 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:22.300340 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f857b8-5678-40ec-aef5-194457fd4261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:22.300908 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f857b8-5678-40ec-aef5-194457fd4261 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:22.301208 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f857b8-5678-40ec-aef5-194457fd4261 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:22.301488 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f857b8-5678-40ec-aef5-194457fd4261 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:22.317053 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f857b8-5678-40ec-aef5-194457fd4261 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:22.319067 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f857b8-5678-40ec-aef5-194457fd4261 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:22.332346 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 368/728] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:22:22 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:22:22.346729 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-07f857b8-5678-40ec-aef5-194457fd4261 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:22.346729 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:22.346729 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:22.346729 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:22.346729 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:22.346729 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:22.346729 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:22.346729 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:22.346729 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:22.346729 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:22.346729 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:22.346729 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:22.346729 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:22:22.346729 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:22:22.346729 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:22:22.346729 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:22:22.347954 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:22.347954 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:22.347954 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:22.347954 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:22.347954 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:22.347954 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:22.347954 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:22.347954 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:22.347954 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:22.347954 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:22.347954 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:22.347954 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:22.347954 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:22.347954 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:22.347954 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:22.347954 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:22.347954 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:22:22.349080 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:22:22.349080 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:22.349080 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:22.355612 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 361/729] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:22:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:22.357525 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df64c3e-f51a-43db-b9bb-df673944bed6 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:22.492257 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df64c3e-f51a-43db-b9bb-df673944bed6 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:22.496791 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:22.538955 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d050ca0d-8c6a-4b81-8e16-ec30a5357d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:22.539860 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d050ca0d-8c6a-4b81-8e16-ec30a5357d8c None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:22.540856 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d050ca0d-8c6a-4b81-8e16-ec30a5357d8c None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:22.541318 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d050ca0d-8c6a-4b81-8e16-ec30a5357d8c None admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:22.542173 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d050ca0d-8c6a-4b81-8e16-ec30a5357d8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:22.542652 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d050ca0d-8c6a-4b81-8e16-ec30a5357d8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:22.544367 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d050ca0d-8c6a-4b81-8e16-ec30a5357d8c None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:22.561558 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 369/730] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 24 06:22:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:22.564707 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f857b8-5678-40ec-aef5-194457fd4261 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:22.680248 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a21a8c-c2be-4697-a604-a27f46e52773 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:22.681507 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a21a8c-c2be-4697-a604-a27f46e52773 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:22.682021 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a21a8c-c2be-4697-a604-a27f46e52773 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:22.685011 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 370/731] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:22:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 06:22:22.690504 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1110b0e9-665b-41f8-8ac3-ff25b8381246 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:22.690790 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1110b0e9-665b-41f8-8ac3-ff25b8381246 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:22.691067 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1110b0e9-665b-41f8-8ac3-ff25b8381246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:22.721462 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-1110b0e9-665b-41f8-8ac3-ff25b8381246 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:22:22.722680 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-1110b0e9-665b-41f8-8ac3-ff25b8381246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:22:22.731992 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f857b8-5678-40ec-aef5-194457fd4261 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:22.778577 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb37a9f-4ce6-4ee5-b5b9-ba9b24ac5cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:22.779239 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb37a9f-4ce6-4ee5-b5b9-ba9b24ac5cab None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:22.779492 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb37a9f-4ce6-4ee5-b5b9-ba9b24ac5cab None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:22.779762 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb37a9f-4ce6-4ee5-b5b9-ba9b24ac5cab None admin] PATH_INFO: `/v3/users/swift` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:22.791069 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb37a9f-4ce6-4ee5-b5b9-ba9b24ac5cab None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:22.792673 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb37a9f-4ce6-4ee5-b5b9-ba9b24ac5cab None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:22.808919 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-4cb37a9f-4ce6-4ee5-b5b9-ba9b24ac5cab None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 06:22:22.808919 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:22.808919 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:22.808919 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:22.808919 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:22.808919 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:22.808919 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:22.808919 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:22.808919 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:22.808919 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:22.808919 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:22.808919 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:22.808919 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 06:22:22.808919 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 06:22:22.808919 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 06:22:22.808919 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 06:22:22.810135 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:22.810135 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:22.810135 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 06:22:22.810135 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:22:22.810135 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 06:22:22.810135 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:22:22.810135 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:22.810135 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:22.810135 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:22.810135 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:22.810135 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:22.810135 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:22.810135 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:22.810135 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:22.810135 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:22.810135 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:22.810135 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:22.810135 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:22.811089 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:22.811089 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:22.811089 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 06:22:22.811089 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 06:22:22.811089 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 06:22:22.811089 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 06:22:22.811089 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 06:22:22.811089 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 06:22:22.811089 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 06:22:22.811089 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:22.811089 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 362/732] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Aug 24 06:22:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:22.811739 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb37a9f-4ce6-4ee5-b5b9-ba9b24ac5cab None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:22.816480 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 371/733] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:22:22 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 06:22:22.818869 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1110b0e9-665b-41f8-8ac3-ff25b8381246 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:22.844853 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:22.976312 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:22.988922 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1110b0e9-665b-41f8-8ac3-ff25b8381246 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:23.016705 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb37a9f-4ce6-4ee5-b5b9-ba9b24ac5cab None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:23.038027 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd58145-eacf-4671-813e-1dfb0494c99c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:23.039477 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd58145-eacf-4671-813e-1dfb0494c99c None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:23.040720 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd58145-eacf-4671-813e-1dfb0494c99c None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:23.041201 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd58145-eacf-4671-813e-1dfb0494c99c None admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:23.042479 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd58145-eacf-4671-813e-1dfb0494c99c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:23.043391 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd58145-eacf-4671-813e-1dfb0494c99c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:23.045373 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd58145-eacf-4671-813e-1dfb0494c99c None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:23.059665 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2311115c-e093-45b7-ab7f-5eb5bcf1e3c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:23.060714 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2311115c-e093-45b7-ab7f-5eb5bcf1e3c2 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:23.061072 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2311115c-e093-45b7-ab7f-5eb5bcf1e3c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:23.061499 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2311115c-e093-45b7-ab7f-5eb5bcf1e3c2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:23.069806 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 372/734] 10.222.0.47 () {56 vars in 1135 bytes} [Wed Aug 24 06:22:22 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:23.071618 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfc0a26-a8a8-4c51-b069-f386fa6c0317 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:23.072302 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfc0a26-a8a8-4c51-b069-f386fa6c0317 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:23.072950 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfc0a26-a8a8-4c51-b069-f386fa6c0317 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:23.074575 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 373/735] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:22:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:22:23.078913 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfc0a26-a8a8-4c51-b069-f386fa6c0317 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:23.083616 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2311115c-e093-45b7-ab7f-5eb5bcf1e3c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:22:23.091660 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-2311115c-e093-45b7-ab7f-5eb5bcf1e3c2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:22:23.091660 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:23.091660 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:23.091660 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:23.091660 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:23.091660 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:23.091660 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:23.091660 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:23.091660 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:23.091660 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:23.091660 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:23.091660 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:23.091660 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:22:23.091660 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:22:23.091660 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:22:23.091660 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:22:23.092644 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:22:23.092644 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:22:23.092644 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:22:23.092644 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:22:23.092644 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:23.092644 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:23.092644 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:23.092644 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:23.092644 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:23.092644 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:23.092644 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:23.092644 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:23.092644 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:23.092644 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:23.092644 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:23.092644 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:23.093496 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:23.093496 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:23.093496 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:23.093496 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:23.093496 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:22:23.093496 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:22:23.093496 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:22:23.093496 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:22:23.093496 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:22:23.093496 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:22:23.093496 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:22:23.093496 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:23.096519 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 363/736] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Aug 24 06:22:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:23.098288 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dc6966-2f47-4b1e-b60e-bb3083ae5e28 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:23.098715 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dc6966-2f47-4b1e-b60e-bb3083ae5e28 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:23.099126 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dc6966-2f47-4b1e-b60e-bb3083ae5e28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:23.123186 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-f8dc6966-2f47-4b1e-b60e-bb3083ae5e28 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:22:23.124568 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-f8dc6966-2f47-4b1e-b60e-bb3083ae5e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:22:23.212483 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 364/737] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:22:23 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 06:22:23.213515 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dc6966-2f47-4b1e-b60e-bb3083ae5e28 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:23.215984 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:23.222496 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfc0a26-a8a8-4c51-b069-f386fa6c0317 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:23.261330 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69ccad6-42da-4249-83b3-07f90ca1a6b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:23.261976 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69ccad6-42da-4249-83b3-07f90ca1a6b2 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:23.262252 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69ccad6-42da-4249-83b3-07f90ca1a6b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:23.262523 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69ccad6-42da-4249-83b3-07f90ca1a6b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:23.294834 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:23.343385 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dc6966-2f47-4b1e-b60e-bb3083ae5e28 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:23.367410 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69ccad6-42da-4249-83b3-07f90ca1a6b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:23.370145 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69ccad6-42da-4249-83b3-07f90ca1a6b2 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:23.385842 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c46d9ad-bd98-476f-a6df-dc85acc994ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:23.386927 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c46d9ad-bd98-476f-a6df-dc85acc994ad None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:23.387229 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c46d9ad-bd98-476f-a6df-dc85acc994ad None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:23.387673 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c46d9ad-bd98-476f-a6df-dc85acc994ad None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:23.388717 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c46d9ad-bd98-476f-a6df-dc85acc994ad None admin] RBAC: Adding query filter params (name=admin) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:23.389257 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c46d9ad-bd98-476f-a6df-dc85acc994ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:23.391613 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c46d9ad-bd98-476f-a6df-dc85acc994ad None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:23.405630 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 365/738] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 24 06:22:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:23.406551 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c46d9ad-bd98-476f-a6df-dc85acc994ad None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:23.490185 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:23.507761 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 374/739] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 24 06:22:23 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 06:22:23.509124 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69ccad6-42da-4249-83b3-07f90ca1a6b2 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:23.536898 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c46d9ad-bd98-476f-a6df-dc85acc994ad None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:23.574903 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:23.582989 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75b7634-8213-49be-913e-542099371df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:23.585989 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75b7634-8213-49be-913e-542099371df7 None admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:23.585989 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75b7634-8213-49be-913e-542099371df7 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:23.585989 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75b7634-8213-49be-913e-542099371df7 None admin] PATH_INFO: `/v3/services` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:23.585989 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75b7634-8213-49be-913e-542099371df7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:23.586257 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75b7634-8213-49be-913e-542099371df7 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:23.628682 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69ccad6-42da-4249-83b3-07f90ca1a6b2 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:23.654312 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 366/740] 10.222.0.47 () {60 vars in 1141 bytes} [Wed Aug 24 06:22:23 2022] POST /identity/v3/services => generated 240 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:22:23.655624 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75b7634-8213-49be-913e-542099371df7 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:23.687700 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86f2973-c80b-4491-99d6-341b09a221eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:23.688794 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86f2973-c80b-4491-99d6-341b09a221eb None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:23.689307 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86f2973-c80b-4491-99d6-341b09a221eb None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:23.695109 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86f2973-c80b-4491-99d6-341b09a221eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:23.707394 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86f2973-c80b-4491-99d6-341b09a221eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:23.709703 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86f2973-c80b-4491-99d6-341b09a221eb None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:23.734147 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-f86f2973-c80b-4491-99d6-341b09a221eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:23.734147 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:23.734147 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:23.734147 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:23.734147 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:23.734147 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:23.734147 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:23.734147 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:23.734147 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:23.734147 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:23.734147 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:23.734147 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:23.734147 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:22:23.734147 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:22:23.734147 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:22:23.734147 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:22:23.735180 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:23.735180 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:23.735180 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:23.735180 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:23.735180 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:23.735180 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:23.735180 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:23.735180 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:23.735180 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:23.735180 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:23.735180 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:23.735180 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:23.735180 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:23.735180 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:23.735180 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:23.735180 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:23.735180 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:22:23.736099 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:22:23.736099 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:23.736099 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:23.744327 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 375/741] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:22:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:23.744327 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86f2973-c80b-4491-99d6-341b09a221eb None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:23.821073 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75b7634-8213-49be-913e-542099371df7 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:23.871397 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:23.894578 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:23.894782 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a0364f-ac84-4544-a67a-ea26d43c4011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:23.894782 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a0364f-ac84-4544-a67a-ea26d43c4011 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:23.894974 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86f2973-c80b-4491-99d6-341b09a221eb None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:23.895074 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a0364f-ac84-4544-a67a-ea26d43c4011 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:23.895074 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a0364f-ac84-4544-a67a-ea26d43c4011 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:23.917211 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a0364f-ac84-4544-a67a-ea26d43c4011 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:22:23.917211 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-17a0364f-ac84-4544-a67a-ea26d43c4011 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:22:23.917211 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:23.917211 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:23.917211 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:23.917211 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:23.917211 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:23.917211 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:23.917211 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:23.917211 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:23.917211 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:23.917211 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:23.917211 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:23.917211 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:22:23.918309 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:22:23.918309 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:22:23.918309 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:22:23.918309 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:22:23.918309 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:22:23.918309 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:22:23.918309 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:22:23.918309 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:23.918309 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:23.918309 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:23.918309 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:23.918309 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:23.918309 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:23.918309 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:23.918309 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:23.918309 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:23.918309 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:23.919298 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:23.919298 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:23.919298 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:23.919298 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:23.919298 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:23.919298 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:23.919298 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:22:23.919298 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:22:23.919298 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:22:23.919298 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:22:23.919298 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:22:23.919298 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:22:23.919298 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:22:23.919298 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:23.921275 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 367/742] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Aug 24 06:22:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:23.922152 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a0364f-ac84-4544-a67a-ea26d43c4011 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:23.942920 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659caa7f-8d0b-454a-9950-5feab4c4833f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:23.944082 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659caa7f-8d0b-454a-9950-5feab4c4833f None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:23.944469 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659caa7f-8d0b-454a-9950-5feab4c4833f None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:23.944844 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659caa7f-8d0b-454a-9950-5feab4c4833f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:23.962807 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659caa7f-8d0b-454a-9950-5feab4c4833f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:23.964675 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659caa7f-8d0b-454a-9950-5feab4c4833f None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:23.984984 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-659caa7f-8d0b-454a-9950-5feab4c4833f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:23.984984 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:23.984984 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:23.984984 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:23.984984 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:23.984984 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:23.984984 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:23.984984 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:23.984984 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:23.984984 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:23.984984 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:23.984984 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:23.984984 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:22:23.984984 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:22:23.984984 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:22:23.984984 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:22:23.986977 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:23.986977 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:23.986977 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:23.986977 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:23.986977 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:23.986977 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:23.986977 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:23.986977 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:23.986977 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:23.986977 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:23.986977 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:23.986977 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:23.986977 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:23.986977 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:23.986977 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:23.986977 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:23.986977 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:22:23.987986 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:22:23.987986 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:23.987986 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:23.987986 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 376/743] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:22:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:23.987986 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659caa7f-8d0b-454a-9950-5feab4c4833f None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:24.057827 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a0364f-ac84-4544-a67a-ea26d43c4011 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:24.101926 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96f7b6e-f077-435c-84f1-bb45d2951a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:24.103709 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f7b6e-f077-435c-84f1-bb45d2951a11 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:24.104113 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f7b6e-f077-435c-84f1-bb45d2951a11 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:24.104495 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96f7b6e-f077-435c-84f1-bb45d2951a11 None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:24.105593 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96f7b6e-f077-435c-84f1-bb45d2951a11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:24.106011 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96f7b6e-f077-435c-84f1-bb45d2951a11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:24.108332 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96f7b6e-f077-435c-84f1-bb45d2951a11 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:24.116469 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659caa7f-8d0b-454a-9950-5feab4c4833f None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:24.124542 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 368/744] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 24 06:22:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:24.130144 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96f7b6e-f077-435c-84f1-bb45d2951a11 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:24.160159 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf466646-3ed9-4e4d-a963-f6ec4a5666c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:24.161105 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf466646-3ed9-4e4d-a963-f6ec4a5666c4 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:24.161558 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf466646-3ed9-4e4d-a963-f6ec4a5666c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:24.161973 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf466646-3ed9-4e4d-a963-f6ec4a5666c4 None admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:24.162794 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf466646-3ed9-4e4d-a963-f6ec4a5666c4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:24.163240 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf466646-3ed9-4e4d-a963-f6ec4a5666c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:24.164828 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf466646-3ed9-4e4d-a963-f6ec4a5666c4 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:24.178254 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 377/745] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Aug 24 06:22:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:24.180016 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf466646-3ed9-4e4d-a963-f6ec4a5666c4 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:24.188632 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:24.203203 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:24.252241 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96f7b6e-f077-435c-84f1-bb45d2951a11 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:24.291766 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcdbd40-8ab8-48bd-b661-9b99ce585f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:24.292391 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcdbd40-8ab8-48bd-b661-9b99ce585f91 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:24.292672 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcdbd40-8ab8-48bd-b661-9b99ce585f91 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:24.292945 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcdbd40-8ab8-48bd-b661-9b99ce585f91 None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:24.293636 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcdbd40-8ab8-48bd-b661-9b99ce585f91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:24.293937 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcdbd40-8ab8-48bd-b661-9b99ce585f91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:24.295324 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcdbd40-8ab8-48bd-b661-9b99ce585f91 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:24.299928 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf466646-3ed9-4e4d-a963-f6ec4a5666c4 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:24.307456 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 369/746] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 24 06:22:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:24.308732 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50995dd-f363-4c76-9117-37156ddb330c None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:24.309017 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50995dd-f363-4c76-9117-37156ddb330c None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:24.309331 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50995dd-f363-4c76-9117-37156ddb330c None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:24.313279 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 370/747] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:22:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:22:24.313279 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50995dd-f363-4c76-9117-37156ddb330c None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:24.356256 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c25db7-67a4-45a9-8e04-9b23852567bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:24.357033 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c25db7-67a4-45a9-8e04-9b23852567bc None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:24.357613 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c25db7-67a4-45a9-8e04-9b23852567bc None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:24.357803 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c25db7-67a4-45a9-8e04-9b23852567bc None admin] PATH_INFO: `/v3/users/nova` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:24.366396 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c25db7-67a4-45a9-8e04-9b23852567bc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:24.367842 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c25db7-67a4-45a9-8e04-9b23852567bc None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:24.384079 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-83c25db7-67a4-45a9-8e04-9b23852567bc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 06:22:24.384079 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:24.384079 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:24.384079 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:24.384079 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:24.384079 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:24.384079 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:24.384079 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:24.384079 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:24.384079 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:24.384079 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:24.384079 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:24.384079 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 06:22:24.384079 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 06:22:24.384079 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 06:22:24.384079 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 06:22:24.390306 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:24.390306 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:24.390306 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 06:22:24.390306 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:22:24.390306 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 06:22:24.390306 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:22:24.390306 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:24.390306 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:24.390306 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:24.390306 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:24.390306 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:24.390306 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:24.390306 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:24.390306 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:24.390306 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:24.390306 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:24.390306 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:24.390306 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:24.391472 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:24.391472 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:24.391472 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 06:22:24.391472 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 06:22:24.391472 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 06:22:24.391472 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 06:22:24.391472 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 06:22:24.391472 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 06:22:24.391472 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 06:22:24.391472 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:24.391472 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 378/748] 10.222.0.47 () {56 vars in 1093 bytes} [Wed Aug 24 06:22:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:24.391472 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c25db7-67a4-45a9-8e04-9b23852567bc None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:24.465882 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50995dd-f363-4c76-9117-37156ddb330c None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:24.470465 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:24.480322 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:24.510235 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be97283-5091-48c8-998e-f5c3fe98e981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:24.510873 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be97283-5091-48c8-998e-f5c3fe98e981 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:24.511153 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be97283-5091-48c8-998e-f5c3fe98e981 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:24.511432 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be97283-5091-48c8-998e-f5c3fe98e981 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:24.514832 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c25db7-67a4-45a9-8e04-9b23852567bc None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:24.522009 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be97283-5091-48c8-998e-f5c3fe98e981 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:24.523828 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be97283-5091-48c8-998e-f5c3fe98e981 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:24.540267 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-8be97283-5091-48c8-998e-f5c3fe98e981 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:24.540267 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:24.540267 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:24.540267 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:24.540267 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:24.540267 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:24.540267 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:24.540267 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:24.540267 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:24.540267 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:24.540267 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:24.540267 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:24.540267 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:22:24.540267 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:22:24.540267 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:22:24.540267 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:22:24.541442 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:24.541442 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:24.541442 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:24.541442 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:24.541442 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:24.541442 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:24.541442 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:24.541442 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:24.541442 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:24.541442 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:24.541442 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:24.541442 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:24.541442 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:24.541442 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:24.541442 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:24.541442 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:24.541442 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:22:24.542345 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:22:24.542345 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:24.542345 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:24.542345 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 371/749] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:22:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:24.542997 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc32f4ae-3bd7-4b59-b5a1-3ad0fc853974 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:24.543283 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc32f4ae-3bd7-4b59-b5a1-3ad0fc853974 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:24.543553 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc32f4ae-3bd7-4b59-b5a1-3ad0fc853974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:24.559298 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05889339-9b13-495e-9ccf-f515a683e682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:24.560176 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05889339-9b13-495e-9ccf-f515a683e682 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:24.560600 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05889339-9b13-495e-9ccf-f515a683e682 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:24.561020 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05889339-9b13-495e-9ccf-f515a683e682 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:24.562133 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-dc32f4ae-3bd7-4b59-b5a1-3ad0fc853974 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:22:24.563322 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-dc32f4ae-3bd7-4b59-b5a1-3ad0fc853974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:22:24.568432 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05889339-9b13-495e-9ccf-f515a683e682 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:24.570042 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05889339-9b13-495e-9ccf-f515a683e682 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:24.583553 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-05889339-9b13-495e-9ccf-f515a683e682 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:22:24.583553 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:24.583553 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:24.583553 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:24.583553 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:24.583553 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:24.583553 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:24.583553 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:24.583553 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:24.583553 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:24.583553 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:24.583553 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:24.583553 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 06:22:24.583553 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 06:22:24.583553 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 06:22:24.584660 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 06:22:24.584660 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:24.584660 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:24.584660 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:24.584660 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:24.584660 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:24.584660 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:24.584660 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:24.584660 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:24.584660 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:24.584660 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:24.584660 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:24.584660 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:24.584660 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:24.584660 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:24.584660 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:24.584660 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:24.585737 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 06:22:24.585737 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 06:22:24.585737 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:22:24.585737 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:22:24.585737 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:22:24.585737 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:22:24.585737 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:22:24.585737 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:24.592564 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 379/750] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 24 06:22:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:24.592564 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05889339-9b13-495e-9ccf-f515a683e682 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:24.655627 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 372/751] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:22:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:22:24.656511 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc32f4ae-3bd7-4b59-b5a1-3ad0fc853974 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:24.728126 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05889339-9b13-495e-9ccf-f515a683e682 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:24.768788 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896cb344-dad7-4e3f-be02-0565f1d77db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:24.769440 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896cb344-dad7-4e3f-be02-0565f1d77db2 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:24.769724 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896cb344-dad7-4e3f-be02-0565f1d77db2 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:24.769999 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896cb344-dad7-4e3f-be02-0565f1d77db2 None admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:24.770743 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896cb344-dad7-4e3f-be02-0565f1d77db2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:24.771050 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896cb344-dad7-4e3f-be02-0565f1d77db2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:24.772751 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896cb344-dad7-4e3f-be02-0565f1d77db2 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:24.780153 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:24.794305 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:24.802262 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 380/752] 10.222.0.47 () {56 vars in 1133 bytes} [Wed Aug 24 06:22:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:24.802262 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896cb344-dad7-4e3f-be02-0565f1d77db2 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:24.810141 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc32f4ae-3bd7-4b59-b5a1-3ad0fc853974 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:24.860816 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9094492-265e-4dd9-b055-f300c7ff0b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:24.862707 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9094492-265e-4dd9-b055-f300c7ff0b65 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:24.863201 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9094492-265e-4dd9-b055-f300c7ff0b65 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:24.863707 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9094492-265e-4dd9-b055-f300c7ff0b65 None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:24.864587 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9094492-265e-4dd9-b055-f300c7ff0b65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:24.865716 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9094492-265e-4dd9-b055-f300c7ff0b65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:24.867717 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9094492-265e-4dd9-b055-f300c7ff0b65 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:24.886058 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 373/753] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 24 06:22:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:24.887097 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9094492-265e-4dd9-b055-f300c7ff0b65 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:24.939822 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896cb344-dad7-4e3f-be02-0565f1d77db2 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:24.985092 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7606ee-001c-4feb-a992-e534d0186fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:24.985777 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7606ee-001c-4feb-a992-e534d0186fb4 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:24.986162 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7606ee-001c-4feb-a992-e534d0186fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:24.986560 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7606ee-001c-4feb-a992-e534d0186fb4 None admin] PATH_INFO: `/v3/projects` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:24.987795 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7606ee-001c-4feb-a992-e534d0186fb4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:24.988204 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7606ee-001c-4feb-a992-e534d0186fb4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:24.989861 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7606ee-001c-4feb-a992-e534d0186fb4 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:25.001554 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9094492-265e-4dd9-b055-f300c7ff0b65 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:25.003365 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 381/754] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Aug 24 06:22:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:25.004243 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7606ee-001c-4feb-a992-e534d0186fb4 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:25.047238 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909dbb20-29c9-459b-8464-f20b91a79968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:25.047906 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909dbb20-29c9-459b-8464-f20b91a79968 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:25.048193 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909dbb20-29c9-459b-8464-f20b91a79968 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:25.048556 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909dbb20-29c9-459b-8464-f20b91a79968 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:25.065813 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909dbb20-29c9-459b-8464-f20b91a79968 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:25.067277 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909dbb20-29c9-459b-8464-f20b91a79968 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:25.074739 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:25.077806 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:25.083126 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-909dbb20-29c9-459b-8464-f20b91a79968 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:25.083126 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:25.083126 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:25.083126 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:25.083126 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:25.083126 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:25.083126 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:25.083126 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:25.083126 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:25.083126 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:25.083126 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:25.083126 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:25.083126 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:22:25.083126 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:22:25.083126 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:22:25.083126 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:22:25.090950 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:25.090950 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:25.090950 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:25.090950 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:25.090950 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:25.090950 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:25.090950 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:25.090950 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:25.090950 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:25.090950 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:25.090950 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:25.090950 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:25.090950 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:25.090950 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:25.090950 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:25.090950 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:25.090950 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:22:25.092038 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:22:25.092038 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:25.092038 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:25.092038 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 374/755] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:22:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:25.092038 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909dbb20-29c9-459b-8464-f20b91a79968 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:25.138503 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7606ee-001c-4feb-a992-e534d0186fb4 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:25.176379 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d5316a-79ca-4828-821b-9d9e138ebdeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:25.177040 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d5316a-79ca-4828-821b-9d9e138ebdeb None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:25.177354 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d5316a-79ca-4828-821b-9d9e138ebdeb None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:25.177628 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d5316a-79ca-4828-821b-9d9e138ebdeb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:25.186839 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d5316a-79ca-4828-821b-9d9e138ebdeb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:25.188283 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d5316a-79ca-4828-821b-9d9e138ebdeb None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:25.200696 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-21d5316a-79ca-4828-821b-9d9e138ebdeb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 06:22:25.200696 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:25.200696 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:25.200696 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:25.200696 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:25.200696 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:25.200696 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:25.200696 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:25.200696 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:25.200696 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:25.200696 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:25.200696 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:25.200696 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 06:22:25.200696 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 06:22:25.200696 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 06:22:25.200696 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 06:22:25.201849 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:25.201849 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:25.201849 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 06:22:25.201849 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:22:25.201849 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 06:22:25.201849 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:22:25.201849 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:25.201849 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:25.201849 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:25.201849 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:25.201849 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:25.201849 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:25.201849 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:25.201849 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:25.201849 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:25.201849 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:25.201849 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:25.201849 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:25.203156 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:25.203156 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:25.203156 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 06:22:25.203156 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 06:22:25.203156 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 06:22:25.203156 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 06:22:25.203156 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 06:22:25.203156 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 06:22:25.203156 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 06:22:25.203156 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:25.203156 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 382/756] 10.222.0.47 () {56 vars in 1117 bytes} [Wed Aug 24 06:22:25 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:25.206211 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d5316a-79ca-4828-821b-9d9e138ebdeb None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:25.210551 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909dbb20-29c9-459b-8464-f20b91a79968 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:25.249922 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c7f2c3-5062-46d7-b74d-026620f19c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:25.250462 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c7f2c3-5062-46d7-b74d-026620f19c98 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:25.250740 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c7f2c3-5062-46d7-b74d-026620f19c98 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:25.251009 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c7f2c3-5062-46d7-b74d-026620f19c98 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:25.251592 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c7f2c3-5062-46d7-b74d-026620f19c98 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:25.252960 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c7f2c3-5062-46d7-b74d-026620f19c98 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:25.263461 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-67c7f2c3-5062-46d7-b74d-026620f19c98 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 06:22:25.263461 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:25.263461 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 06:22:25.263461 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 06:22:25.263461 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 06:22:25.263461 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 06:22:25.263461 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 06:22:25.263461 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:22:25.263461 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 06:22:25.263461 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:25.263461 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 06:22:25.263461 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:25.263461 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:25.263461 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:25.263461 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:25.263461 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:25.263461 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:25.265745 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:25.265745 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:25.265745 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:25.265745 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:25.265745 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:25.265745 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:25.265745 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 06:22:25.265745 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 06:22:25.265745 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 06:22:25.265745 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 06:22:25.265745 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:25.265745 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:25.265745 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:25.265745 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:25.265745 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:25.265745 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:25.267271 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:25.267271 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:25.267271 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:25.267271 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:25.267271 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:25.267271 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:25.267271 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:25.267271 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:25.267271 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:25.267271 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:25.267271 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 06:22:25.267271 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:22:25.267271 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 06:22:25.267271 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:25.269496 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 375/757] 10.222.0.47 () {56 vars in 1129 bytes} [Wed Aug 24 06:22:25 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:25.270463 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c7f2c3-5062-46d7-b74d-026620f19c98 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:25.345498 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:25.346303 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d5316a-79ca-4828-821b-9d9e138ebdeb None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:25.364340 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:25.393020 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9a0fac-cac3-4c57-a1f2-785f1cb592d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:25.393020 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9a0fac-cac3-4c57-a1f2-785f1cb592d8 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:25.393020 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9a0fac-cac3-4c57-a1f2-785f1cb592d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:25.393020 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9a0fac-cac3-4c57-a1f2-785f1cb592d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:25.393524 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9a0fac-cac3-4c57-a1f2-785f1cb592d8 None admin] RBAC: Adding query filter params (role.id=486144ad1bcd417e86acdbc1d14327bf, scope.project.id=c02e49a89cdd40d885bfa816f06c58a1, user.id=02f1cbaf2cd6427585f298c9d58c319a) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:25.393524 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9a0fac-cac3-4c57-a1f2-785f1cb592d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:25.393524 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9a0fac-cac3-4c57-a1f2-785f1cb592d8 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:25.398501 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c7f2c3-5062-46d7-b74d-026620f19c98 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:25.409535 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 383/758] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 24 06:22:25 2022] GET /identity/v3/role_assignments?user.id=02f1cbaf2cd6427585f298c9d58c319a&scope.project.id=c02e49a89cdd40d885bfa816f06c58a1&role.id=486144ad1bcd417e86acdbc1d14327bf => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:25.409535 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9a0fac-cac3-4c57-a1f2-785f1cb592d8 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:25.453407 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0916f6b-2113-4421-b8c4-95be767dcb4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:25.454205 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0916f6b-2113-4421-b8c4-95be767dcb4d None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:25.454580 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0916f6b-2113-4421-b8c4-95be767dcb4d None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:25.454947 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0916f6b-2113-4421-b8c4-95be767dcb4d None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:25.455855 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0916f6b-2113-4421-b8c4-95be767dcb4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:25.456334 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0916f6b-2113-4421-b8c4-95be767dcb4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:25.457888 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0916f6b-2113-4421-b8c4-95be767dcb4d None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:25.477299 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 376/759] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 24 06:22:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:25.479299 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0916f6b-2113-4421-b8c4-95be767dcb4d None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:25.566842 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9a0fac-cac3-4c57-a1f2-785f1cb592d8 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:25.614992 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae0a03a-7425-48f9-9c8b-88ffe3d5676a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:25.615698 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae0a03a-7425-48f9-9c8b-88ffe3d5676a None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:25.616045 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae0a03a-7425-48f9-9c8b-88ffe3d5676a None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:25.616272 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0916f6b-2113-4421-b8c4-95be767dcb4d None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:25.616381 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae0a03a-7425-48f9-9c8b-88ffe3d5676a None admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:25.617201 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae0a03a-7425-48f9-9c8b-88ffe3d5676a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:25.617683 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae0a03a-7425-48f9-9c8b-88ffe3d5676a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:25.620727 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae0a03a-7425-48f9-9c8b-88ffe3d5676a None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:25.653880 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 384/760] 10.222.0.47 () {56 vars in 1149 bytes} [Wed Aug 24 06:22:25 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:25.655556 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae0a03a-7425-48f9-9c8b-88ffe3d5676a None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:25.666042 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:25.679313 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5e897c-97e6-41f3-a9bf-8b8018eaabbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:25.680176 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5e897c-97e6-41f3-a9bf-8b8018eaabbd None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:25.680454 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5e897c-97e6-41f3-a9bf-8b8018eaabbd None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:25.680763 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5e897c-97e6-41f3-a9bf-8b8018eaabbd None admin] PATH_INFO: `/v3/services` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:25.681951 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5e897c-97e6-41f3-a9bf-8b8018eaabbd None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:25.681951 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5e897c-97e6-41f3-a9bf-8b8018eaabbd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:25.683456 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5e897c-97e6-41f3-a9bf-8b8018eaabbd None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:25.693725 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 377/761] 10.222.0.47 () {56 vars in 1124 bytes} [Wed Aug 24 06:22:25 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:25.695187 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5e897c-97e6-41f3-a9bf-8b8018eaabbd None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:25.705358 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:25.796694 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae0a03a-7425-48f9-9c8b-88ffe3d5676a None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:25.827574 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5e897c-97e6-41f3-a9bf-8b8018eaabbd None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:25.840680 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2b53fb-2693-47d0-949b-b1ef03a116b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:25.841362 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2b53fb-2693-47d0-949b-b1ef03a116b1 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:25.841647 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2b53fb-2693-47d0-949b-b1ef03a116b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:25.841921 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2b53fb-2693-47d0-949b-b1ef03a116b1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:25.851084 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2b53fb-2693-47d0-949b-b1ef03a116b1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:25.852728 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2b53fb-2693-47d0-949b-b1ef03a116b1 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:25.866073 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-cd2b53fb-2693-47d0-949b-b1ef03a116b1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:22:25.866073 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:25.866073 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:25.866073 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:25.866073 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:25.866073 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:25.866073 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:25.866073 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:25.866073 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:25.866073 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:25.866073 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:25.866073 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:25.866073 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 06:22:25.866073 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 06:22:25.866073 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 06:22:25.867126 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 06:22:25.867126 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:25.867126 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:25.867126 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:25.867126 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:25.867126 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:25.867126 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:25.867126 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:25.867126 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:25.867126 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:25.867126 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:25.867126 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:25.867126 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:25.867126 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:25.867126 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:25.867126 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:25.867126 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:25.868021 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 06:22:25.868021 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 06:22:25.868021 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:22:25.868021 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:22:25.868021 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:22:25.868021 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:22:25.868021 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:22:25.868021 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:25.868021 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 385/762] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 24 06:22:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:25.869408 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2b53fb-2693-47d0-949b-b1ef03a116b1 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:25.880771 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2200f13f-4b6a-4321-927d-3103a7cbebe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:25.881488 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2200f13f-4b6a-4321-927d-3103a7cbebe1 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:25.881719 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2200f13f-4b6a-4321-927d-3103a7cbebe1 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:25.881993 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2200f13f-4b6a-4321-927d-3103a7cbebe1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:25.949968 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:25.968684 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2200f13f-4b6a-4321-927d-3103a7cbebe1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:25.970376 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2200f13f-4b6a-4321-927d-3103a7cbebe1 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:25.989480 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:25.999712 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2b53fb-2693-47d0-949b-b1ef03a116b1 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:26.042581 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7f147b-af87-4e5f-9224-a77303c6a560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:26.043479 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f147b-af87-4e5f-9224-a77303c6a560 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:26.043779 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f147b-af87-4e5f-9224-a77303c6a560 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:26.044068 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f147b-af87-4e5f-9224-a77303c6a560 None admin] PATH_INFO: `/v3/services` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:26.044703 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7f147b-af87-4e5f-9224-a77303c6a560 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:26.045004 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7f147b-af87-4e5f-9224-a77303c6a560 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:26.047146 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7f147b-af87-4e5f-9224-a77303c6a560 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:26.057123 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 386/763] 10.222.0.47 () {56 vars in 1124 bytes} [Wed Aug 24 06:22:25 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:26.058363 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc6c9a8-9679-4a05-9325-220ffd39ed8e None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:26.058656 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc6c9a8-9679-4a05-9325-220ffd39ed8e None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:26.058930 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc6c9a8-9679-4a05-9325-220ffd39ed8e None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:26.064334 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 387/764] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:22:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:22:26.064334 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc6c9a8-9679-4a05-9325-220ffd39ed8e None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:26.096240 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 378/765] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 24 06:22:25 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 06:22:26.096240 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2200f13f-4b6a-4321-927d-3103a7cbebe1 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:26.198445 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc6c9a8-9679-4a05-9325-220ffd39ed8e None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:26.200336 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:26.218989 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2200f13f-4b6a-4321-927d-3103a7cbebe1 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:26.235283 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2ffbad-5bb6-4cd0-8667-fa6e84b87ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:26.235900 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2ffbad-5bb6-4cd0-8667-fa6e84b87ac0 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:26.236176 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2ffbad-5bb6-4cd0-8667-fa6e84b87ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:26.236449 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2ffbad-5bb6-4cd0-8667-fa6e84b87ac0 None admin] PATH_INFO: `/v3/projects` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:26.237142 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2ffbad-5bb6-4cd0-8667-fa6e84b87ac0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:26.237474 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2ffbad-5bb6-4cd0-8667-fa6e84b87ac0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:26.238905 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2ffbad-5bb6-4cd0-8667-fa6e84b87ac0 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:26.257040 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 388/766] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Aug 24 06:22:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:26.257040 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82787fc4-a3f7-49e1-b5f3-937caeac76c1 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:26.257040 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82787fc4-a3f7-49e1-b5f3-937caeac76c1 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:26.257040 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82787fc4-a3f7-49e1-b5f3-937caeac76c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:26.266631 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b31f5a-85f1-4282-8e4c-9c2164bea1f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:26.267256 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b31f5a-85f1-4282-8e4c-9c2164bea1f8 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:26.267543 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b31f5a-85f1-4282-8e4c-9c2164bea1f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:26.267818 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b31f5a-85f1-4282-8e4c-9c2164bea1f8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:26.268535 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b31f5a-85f1-4282-8e4c-9c2164bea1f8 None admin] RBAC: Adding query filter params (interface=public, service_id=6cff02b429ab43f1beb006c54b3cc190, region_id=RegionOne) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:26.268842 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b31f5a-85f1-4282-8e4c-9c2164bea1f8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:26.270256 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b31f5a-85f1-4282-8e4c-9c2164bea1f8 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:26.274081 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-82787fc4-a3f7-49e1-b5f3-937caeac76c1 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:22:26.276589 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-82787fc4-a3f7-49e1-b5f3-937caeac76c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:22:26.278706 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 379/767] 10.222.0.47 () {56 vars in 1251 bytes} [Wed Aug 24 06:22:26 2022] GET /identity/v3/endpoints?service_id=6cff02b429ab43f1beb006c54b3cc190&interface=public®ion_id=RegionOne => generated 194 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:26.279384 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b31f5a-85f1-4282-8e4c-9c2164bea1f8 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:26.295964 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:26.365258 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 389/768] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:22:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:22:26.365258 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82787fc4-a3f7-49e1-b5f3-937caeac76c1 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:26.411810 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b31f5a-85f1-4282-8e4c-9c2164bea1f8 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:26.448709 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5df583-dffc-42e8-a563-bda90b08f4ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:26.449486 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5df583-dffc-42e8-a563-bda90b08f4ad None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:26.449832 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5df583-dffc-42e8-a563-bda90b08f4ad None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:26.450168 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5df583-dffc-42e8-a563-bda90b08f4ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:26.461777 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5df583-dffc-42e8-a563-bda90b08f4ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:26.463971 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5df583-dffc-42e8-a563-bda90b08f4ad None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:26.481516 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-3e5df583-dffc-42e8-a563-bda90b08f4ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:26.481516 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:26.481516 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:26.481516 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:26.481516 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:26.481516 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:26.481516 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:26.481516 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:26.481516 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:26.481516 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:26.481516 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:26.481516 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:26.481516 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:22:26.481516 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:22:26.481516 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:22:26.481516 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:22:26.482555 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:26.482555 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:26.482555 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:26.482555 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:26.482555 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:26.482555 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:26.482555 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:26.482555 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:26.482555 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:26.482555 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:26.482555 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:26.482555 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:26.482555 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:26.482555 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:26.482555 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:26.482555 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:26.482555 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:22:26.483784 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:22:26.483784 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:26.483784 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:26.487303 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 380/769] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:22:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:26.488272 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5df583-dffc-42e8-a563-bda90b08f4ad None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:26.491204 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:26.492401 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82787fc4-a3f7-49e1-b5f3-937caeac76c1 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:26.535146 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d6f3ac-235f-429b-bdb6-bdc39a9a903a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:26.536212 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d6f3ac-235f-429b-bdb6-bdc39a9a903a None admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:26.536746 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d6f3ac-235f-429b-bdb6-bdc39a9a903a None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:26.537319 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d6f3ac-235f-429b-bdb6-bdc39a9a903a None admin] PATH_INFO: `/v3/projects/c02e49a89cdd40d885bfa816f06c58a1/users/1f4f49c1b08c460f8ba6c451c9c92c2a/roles/486144ad1bcd417e86acdbc1d14327bf` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:26.539516 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d6f3ac-235f-429b-bdb6-bdc39a9a903a None admin] RBAC: Authorizing `identity:create_grant(project_id=c02e49a89cdd40d885bfa816f06c58a1, user_id=1f4f49c1b08c460f8ba6c451c9c92c2a, role_id=486144ad1bcd417e86acdbc1d14327bf)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:26.541976 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d6f3ac-235f-429b-bdb6-bdc39a9a903a None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:26.624699 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5df583-dffc-42e8-a563-bda90b08f4ad None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:26.655106 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 390/770] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Aug 24 06:22:26 2022] PUT /identity/v3/projects/c02e49a89cdd40d885bfa816f06c58a1/users/1f4f49c1b08c460f8ba6c451c9c92c2a/roles/486144ad1bcd417e86acdbc1d14327bf => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:22:26.656843 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d6f3ac-235f-429b-bdb6-bdc39a9a903a None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:26.673231 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:26.679291 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f813ee-8667-4a40-9dce-7fed29fbdb6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:26.679990 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f813ee-8667-4a40-9dce-7fed29fbdb6b None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:26.680438 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f813ee-8667-4a40-9dce-7fed29fbdb6b None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:26.680583 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f813ee-8667-4a40-9dce-7fed29fbdb6b None admin] PATH_INFO: `/v3/services` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:26.681450 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f813ee-8667-4a40-9dce-7fed29fbdb6b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:26.683498 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f813ee-8667-4a40-9dce-7fed29fbdb6b None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:26.692566 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 381/771] 10.222.0.47 () {56 vars in 1087 bytes} [Wed Aug 24 06:22:26 2022] GET /identity/v3/services => generated 772 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:26.693591 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f813ee-8667-4a40-9dce-7fed29fbdb6b None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:26.786853 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:26.867825 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d6f3ac-235f-429b-bdb6-bdc39a9a903a None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:26.916299 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bb3d5e-e644-4658-b041-142aadaf8e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:26.916824 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bb3d5e-e644-4658-b041-142aadaf8e74 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:26.917105 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bb3d5e-e644-4658-b041-142aadaf8e74 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:26.917600 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bb3d5e-e644-4658-b041-142aadaf8e74 None admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:26.918275 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bb3d5e-e644-4658-b041-142aadaf8e74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:26.918581 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bb3d5e-e644-4658-b041-142aadaf8e74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:26.920087 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bb3d5e-e644-4658-b041-142aadaf8e74 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:26.937017 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 391/772] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 24 06:22:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:26.946805 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bb3d5e-e644-4658-b041-142aadaf8e74 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:26.985286 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f813ee-8667-4a40-9dce-7fed29fbdb6b None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:27.021217 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80a6f5f-35e7-46db-b119-019f7f4809b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:27.021861 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80a6f5f-35e7-46db-b119-019f7f4809b0 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:27.022390 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80a6f5f-35e7-46db-b119-019f7f4809b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:27.022663 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80a6f5f-35e7-46db-b119-019f7f4809b0 None admin] PATH_INFO: `/v3/services/network` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:27.023248 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80a6f5f-35e7-46db-b119-019f7f4809b0 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:27.024742 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80a6f5f-35e7-46db-b119-019f7f4809b0 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:27.036407 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-d80a6f5f-35e7-46db-b119-019f7f4809b0 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 06:22:27.036407 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:27.036407 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 06:22:27.036407 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 06:22:27.036407 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 06:22:27.036407 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 06:22:27.036407 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 06:22:27.036407 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:22:27.036407 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 06:22:27.036407 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:27.036407 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 06:22:27.036407 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:27.036407 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:27.036407 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:27.036407 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:27.036407 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:27.036407 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:27.037469 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:27.037469 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:27.037469 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:27.037469 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:27.037469 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:27.037469 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:27.037469 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 06:22:27.037469 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 06:22:27.037469 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 06:22:27.037469 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 06:22:27.037469 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:27.037469 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:27.037469 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:27.037469 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:27.037469 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:27.037469 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:27.038319 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:27.038319 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:27.038319 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:27.038319 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:27.038319 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:27.038319 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:27.038319 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:27.038319 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:27.038319 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:27.038319 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:27.038319 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 06:22:27.038319 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:22:27.038319 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 06:22:27.038319 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:27.038319 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 382/773] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 24 06:22:26 2022] GET /identity/v3/services/network => generated 88 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:27.045222 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80a6f5f-35e7-46db-b119-019f7f4809b0 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:27.057583 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:27.138102 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bb3d5e-e644-4658-b041-142aadaf8e74 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:27.182132 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80a6f5f-35e7-46db-b119-019f7f4809b0 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:27.226210 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:27.238629 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec39626b-9240-4359-8e81-ef082e438f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:27.239254 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec39626b-9240-4359-8e81-ef082e438f66 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:27.239530 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec39626b-9240-4359-8e81-ef082e438f66 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:27.239797 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec39626b-9240-4359-8e81-ef082e438f66 None admin] PATH_INFO: `/v3/services` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:27.240437 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec39626b-9240-4359-8e81-ef082e438f66 None admin] RBAC: Adding query filter params (name=network) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:27.240735 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec39626b-9240-4359-8e81-ef082e438f66 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:27.242135 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec39626b-9240-4359-8e81-ef082e438f66 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:27.247374 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1c5bb0-b15c-4708-9680-d8cbfeac4448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:27.248026 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1c5bb0-b15c-4708-9680-d8cbfeac4448 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:27.248305 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1c5bb0-b15c-4708-9680-d8cbfeac4448 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:27.248576 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1c5bb0-b15c-4708-9680-d8cbfeac4448 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:27.261576 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 383/774] 10.222.0.47 () {56 vars in 1112 bytes} [Wed Aug 24 06:22:27 2022] GET /identity/v3/services?name=network => generated 124 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:27.263536 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec39626b-9240-4359-8e81-ef082e438f66 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:27.264000 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1c5bb0-b15c-4708-9680-d8cbfeac4448 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:27.265504 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1c5bb0-b15c-4708-9680-d8cbfeac4448 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:27.287230 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-fd1c5bb0-b15c-4708-9680-d8cbfeac4448 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:22:27.287230 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:27.287230 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:27.287230 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:27.287230 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:27.287230 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:27.287230 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:27.287230 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:27.287230 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:27.287230 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:27.287230 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:27.287230 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:27.287230 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 06:22:27.287230 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 06:22:27.287230 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 06:22:27.288184 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 06:22:27.288184 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:27.288184 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:27.288184 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:27.288184 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:27.288184 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:27.288184 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:27.288184 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:27.288184 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:27.288184 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:27.288184 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:27.288184 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:27.288184 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:27.288184 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:27.288184 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:27.288184 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:27.288184 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:27.289706 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 06:22:27.289706 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 06:22:27.289706 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:22:27.289706 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:22:27.289706 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:22:27.289706 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:22:27.289706 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:22:27.289706 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:27.289706 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 392/775] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 24 06:22:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:22:27.295722 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1c5bb0-b15c-4708-9680-d8cbfeac4448 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:27.421016 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec39626b-9240-4359-8e81-ef082e438f66 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:27.481022 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4401be9-3b7c-4adc-813c-9830fb9c44a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:27.481578 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4401be9-3b7c-4adc-813c-9830fb9c44a0 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:27.481854 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4401be9-3b7c-4adc-813c-9830fb9c44a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:27.482129 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4401be9-3b7c-4adc-813c-9830fb9c44a0 None admin] PATH_INFO: `/v3/services` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:27.482762 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4401be9-3b7c-4adc-813c-9830fb9c44a0 None admin] RBAC: Adding query filter params (type=network) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:27.483058 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4401be9-3b7c-4adc-813c-9830fb9c44a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:27.490129 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4401be9-3b7c-4adc-813c-9830fb9c44a0 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:27.499206 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 384/776] 10.222.0.47 () {56 vars in 1112 bytes} [Wed Aug 24 06:22:27 2022] GET /identity/v3/services?type=network => generated 350 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:27.504999 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4401be9-3b7c-4adc-813c-9830fb9c44a0 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:27.513357 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1c5bb0-b15c-4708-9680-d8cbfeac4448 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:27.547951 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:27.555830 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b823311-fcb7-4289-b5f3-cafd83d82109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:27.556436 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b823311-fcb7-4289-b5f3-cafd83d82109 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:27.556713 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b823311-fcb7-4289-b5f3-cafd83d82109 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:27.556985 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b823311-fcb7-4289-b5f3-cafd83d82109 None admin] PATH_INFO: `/v3/projects` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:27.557699 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b823311-fcb7-4289-b5f3-cafd83d82109 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:27.558003 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b823311-fcb7-4289-b5f3-cafd83d82109 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:27.559450 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b823311-fcb7-4289-b5f3-cafd83d82109 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:27.572772 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 393/777] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Aug 24 06:22:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:22:27.578981 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c528eb-af2b-46d4-9e4f-e8cb0028a5ea None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:27.579273 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c528eb-af2b-46d4-9e4f-e8cb0028a5ea None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:27.579546 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c528eb-af2b-46d4-9e4f-e8cb0028a5ea None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:27.581131 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 394/778] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:22:27 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:22:27.581885 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c528eb-af2b-46d4-9e4f-e8cb0028a5ea None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:27.588408 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:27.634814 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4401be9-3b7c-4adc-813c-9830fb9c44a0 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:27.671546 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe96c67-74de-4804-abd3-ccb8e7e29393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:27.672288 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe96c67-74de-4804-abd3-ccb8e7e29393 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:27.672570 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe96c67-74de-4804-abd3-ccb8e7e29393 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:27.672846 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe96c67-74de-4804-abd3-ccb8e7e29393 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:27.673604 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe96c67-74de-4804-abd3-ccb8e7e29393 None admin] RBAC: Adding query filter params (interface=public, service_id=a7320cb5337b4cd2aebc5924d22e232a, region_id=RegionOne) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:27.673918 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe96c67-74de-4804-abd3-ccb8e7e29393 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:27.675518 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe96c67-74de-4804-abd3-ccb8e7e29393 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:27.681311 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 385/779] 10.222.0.47 () {56 vars in 1251 bytes} [Wed Aug 24 06:22:27 2022] GET /identity/v3/endpoints?service_id=a7320cb5337b4cd2aebc5924d22e232a&interface=public®ion_id=RegionOne => generated 194 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:27.682560 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e5bc0-b2cd-4c43-b460-9e1e10daa359 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:27.682852 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e5bc0-b2cd-4c43-b460-9e1e10daa359 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:27.683128 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e5bc0-b2cd-4c43-b460-9e1e10daa359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:27.696321 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c528eb-af2b-46d4-9e4f-e8cb0028a5ea None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:27.703095 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-333e5bc0-b2cd-4c43-b460-9e1e10daa359 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:22:27.704431 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-333e5bc0-b2cd-4c43-b460-9e1e10daa359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:22:27.734888 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95544aa8-bc02-4bee-bf4b-12fa36874aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:27.735466 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95544aa8-bc02-4bee-bf4b-12fa36874aed None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:27.735743 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95544aa8-bc02-4bee-bf4b-12fa36874aed None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:27.736012 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95544aa8-bc02-4bee-bf4b-12fa36874aed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:27.736782 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95544aa8-bc02-4bee-bf4b-12fa36874aed None admin] RBAC: Adding query filter params (role.id=1712d85aedaf4189bf4bc89094d888c3, scope.project.id=c02e49a89cdd40d885bfa816f06c58a1, user.id=ea70acd744994250a2905b5dd7163dfb) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:27.737087 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95544aa8-bc02-4bee-bf4b-12fa36874aed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:27.738647 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95544aa8-bc02-4bee-bf4b-12fa36874aed None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:27.753498 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 395/780] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 24 06:22:27 2022] GET /identity/v3/role_assignments?user.id=ea70acd744994250a2905b5dd7163dfb&scope.project.id=c02e49a89cdd40d885bfa816f06c58a1&role.id=1712d85aedaf4189bf4bc89094d888c3 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:27.754545 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95544aa8-bc02-4bee-bf4b-12fa36874aed None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:27.789314 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 386/781] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:22:27 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:22:27.842982 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:27.912130 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95544aa8-bc02-4bee-bf4b-12fa36874aed None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:27.962635 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eacda7-0677-4686-a960-515bef336e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:27.963538 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eacda7-0677-4686-a960-515bef336e57 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:27.963981 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eacda7-0677-4686-a960-515bef336e57 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:27.964401 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eacda7-0677-4686-a960-515bef336e57 None admin] PATH_INFO: `/v3/services` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:27.965173 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eacda7-0677-4686-a960-515bef336e57 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:27.966716 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eacda7-0677-4686-a960-515bef336e57 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:27.978684 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 396/782] 10.222.0.47 () {56 vars in 1087 bytes} [Wed Aug 24 06:22:27 2022] GET /identity/v3/services => generated 772 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:28.169858 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333e5bc0-b2cd-4c43-b460-9e1e10daa359 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:28.229009 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:28.278750 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333e5bc0-b2cd-4c43-b460-9e1e10daa359 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:28.316142 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6398b476-8f5c-49e1-a94e-26c2f3d4cc3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:28.316852 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6398b476-8f5c-49e1-a94e-26c2f3d4cc3d None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:28.317135 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6398b476-8f5c-49e1-a94e-26c2f3d4cc3d None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:28.317441 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6398b476-8f5c-49e1-a94e-26c2f3d4cc3d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:28.330037 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6398b476-8f5c-49e1-a94e-26c2f3d4cc3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:22:28.334902 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-6398b476-8f5c-49e1-a94e-26c2f3d4cc3d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:22:28.334902 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:28.334902 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:28.334902 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:28.334902 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:28.334902 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:28.334902 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:28.334902 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:28.334902 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:28.334902 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:28.334902 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:28.334902 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:28.334902 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:22:28.334902 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:22:28.334902 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:22:28.334902 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:22:28.335834 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:22:28.335834 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:22:28.335834 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:22:28.335834 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:22:28.335834 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:28.335834 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:28.335834 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:28.335834 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:28.335834 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:28.335834 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:28.335834 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:28.335834 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:28.335834 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:28.335834 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:28.335834 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:28.335834 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:28.336843 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:28.336843 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:28.336843 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:28.336843 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:28.336843 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:22:28.336843 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:22:28.336843 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:22:28.336843 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:22:28.336843 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:22:28.336843 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:22:28.336843 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:22:28.336843 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:28.336843 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 387/783] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Aug 24 06:22:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:28.344589 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eacda7-0677-4686-a960-515bef336e57 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:28.507414 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eacda7-0677-4686-a960-515bef336e57 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:28.544117 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:28.568557 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c439b9-f675-4031-a895-95b2cf8a6064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:28.569223 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c439b9-f675-4031-a895-95b2cf8a6064 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:28.573674 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c439b9-f675-4031-a895-95b2cf8a6064 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:28.573674 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c439b9-f675-4031-a895-95b2cf8a6064 None admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:28.573674 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c439b9-f675-4031-a895-95b2cf8a6064 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:28.573674 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c439b9-f675-4031-a895-95b2cf8a6064 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:28.573674 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c439b9-f675-4031-a895-95b2cf8a6064 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:28.585686 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 397/784] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 24 06:22:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:28.592318 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6398b476-8f5c-49e1-a94e-26c2f3d4cc3d None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:28.762159 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6398b476-8f5c-49e1-a94e-26c2f3d4cc3d None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:28.787656 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:28.818844 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e0245e-6290-4073-8b05-a6ba8c6123bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:28.819505 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e0245e-6290-4073-8b05-a6ba8c6123bd None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:28.819790 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e0245e-6290-4073-8b05-a6ba8c6123bd None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:28.820066 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e0245e-6290-4073-8b05-a6ba8c6123bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:28.838543 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e0245e-6290-4073-8b05-a6ba8c6123bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:28.840060 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e0245e-6290-4073-8b05-a6ba8c6123bd None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:28.864116 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-c2e0245e-6290-4073-8b05-a6ba8c6123bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:28.864116 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:28.864116 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:28.864116 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:28.864116 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:28.864116 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:28.864116 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:28.864116 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:28.864116 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:28.864116 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:28.864116 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:28.864116 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:28.864116 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:22:28.864116 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:22:28.864116 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:22:28.864116 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:22:28.865133 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:28.865133 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:28.865133 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:28.865133 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:28.865133 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:28.865133 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:28.865133 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:28.865133 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:28.865133 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:28.865133 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:28.865133 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:28.865133 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:28.865133 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:28.865133 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:28.865133 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:28.865133 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:28.865133 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:22:28.866093 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:22:28.866093 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:28.866093 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:28.866093 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 388/785] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:22:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:28.872702 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c439b9-f675-4031-a895-95b2cf8a6064 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:29.080187 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c439b9-f675-4031-a895-95b2cf8a6064 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:29.105764 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8504c8f2-b942-43d6-8ea0-97908c0e9122 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:29.106098 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8504c8f2-b942-43d6-8ea0-97908c0e9122 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:29.106327 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8504c8f2-b942-43d6-8ea0-97908c0e9122 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:29.109393 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 389/786] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:22:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:22:29.113977 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4ff2a7-5b95-4667-aeea-85dc84eb8e97 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:29.114264 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4ff2a7-5b95-4667-aeea-85dc84eb8e97 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:29.114541 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4ff2a7-5b95-4667-aeea-85dc84eb8e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:29.138011 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2e9c5e-e4c1-4419-9a88-d84e69cb0840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:29.138640 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2e9c5e-e4c1-4419-9a88-d84e69cb0840 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:29.138922 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2e9c5e-e4c1-4419-9a88-d84e69cb0840 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:29.139196 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2e9c5e-e4c1-4419-9a88-d84e69cb0840 None admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:29.139856 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2e9c5e-e4c1-4419-9a88-d84e69cb0840 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:29.140155 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2e9c5e-e4c1-4419-9a88-d84e69cb0840 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:29.142556 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2e9c5e-e4c1-4419-9a88-d84e69cb0840 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:29.148151 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-cf4ff2a7-5b95-4667-aeea-85dc84eb8e97 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:22:29.150041 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-cf4ff2a7-5b95-4667-aeea-85dc84eb8e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:22:29.162761 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 398/787] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 24 06:22:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:29.172906 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2e9c5e-e4c1-4419-9a88-d84e69cb0840 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:29.297386 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 390/788] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:22:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 06:22:29.299107 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba40311-90da-4e53-b918-f01e039c29aa None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:29.299601 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba40311-90da-4e53-b918-f01e039c29aa None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:29.300085 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba40311-90da-4e53-b918-f01e039c29aa None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:29.307467 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 391/789] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:22:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:22:29.319440 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5236311e-ed5d-442d-80fa-050e0dc5fee8 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:29.319783 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5236311e-ed5d-442d-80fa-050e0dc5fee8 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:29.319996 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5236311e-ed5d-442d-80fa-050e0dc5fee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:29.355369 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-5236311e-ed5d-442d-80fa-050e0dc5fee8 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:22:29.357015 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-5236311e-ed5d-442d-80fa-050e0dc5fee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:22:29.403251 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:29.474338 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2e9c5e-e4c1-4419-9a88-d84e69cb0840 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:29.514629 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f98c628-dab7-4496-9a6d-20b354bcf8d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:29.514994 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 392/790] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:22:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 06:22:29.520033 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f98c628-dab7-4496-9a6d-20b354bcf8d4 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:29.520354 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f98c628-dab7-4496-9a6d-20b354bcf8d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:29.520678 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f98c628-dab7-4496-9a6d-20b354bcf8d4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:29.534260 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f98c628-dab7-4496-9a6d-20b354bcf8d4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:29.535752 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f98c628-dab7-4496-9a6d-20b354bcf8d4 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:29.550311 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-0f98c628-dab7-4496-9a6d-20b354bcf8d4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 06:22:29.550311 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:29.550311 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:29.550311 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:29.550311 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:29.550311 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:29.550311 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:29.550311 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:29.550311 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:29.550311 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:29.550311 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:29.550311 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:29.550311 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 06:22:29.550311 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 06:22:29.550311 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 06:22:29.550311 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 06:22:29.551347 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:29.551347 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:29.551347 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 06:22:29.551347 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:22:29.551347 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 06:22:29.551347 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:22:29.551347 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:29.551347 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:29.551347 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:29.551347 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:29.551347 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:29.551347 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:29.551347 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:29.551347 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:29.551347 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:29.551347 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:29.551347 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:29.551347 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:29.552290 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:29.552290 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:29.552290 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 06:22:29.552290 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 06:22:29.552290 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 06:22:29.552290 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 06:22:29.552290 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 06:22:29.552290 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 06:22:29.552290 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 06:22:29.552290 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:29.552290 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 399/791] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Aug 24 06:22:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 379 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:29.560812 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5236311e-ed5d-442d-80fa-050e0dc5fee8 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:29.624378 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:29.742550 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5236311e-ed5d-442d-80fa-050e0dc5fee8 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:29.778540 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f98c628-dab7-4496-9a6d-20b354bcf8d4 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:29.792025 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf41dcc-a39a-4ad6-b275-2a395b0e0dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:29.792841 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf41dcc-a39a-4ad6-b275-2a395b0e0dcb None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:29.793214 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf41dcc-a39a-4ad6-b275-2a395b0e0dcb None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:29.793560 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf41dcc-a39a-4ad6-b275-2a395b0e0dcb None admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:29.794620 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf41dcc-a39a-4ad6-b275-2a395b0e0dcb None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:29.794997 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf41dcc-a39a-4ad6-b275-2a395b0e0dcb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:29.796839 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf41dcc-a39a-4ad6-b275-2a395b0e0dcb None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:29.829136 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 393/792] 10.222.0.47 () {56 vars in 1135 bytes} [Wed Aug 24 06:22:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:29.834779 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf41dcc-a39a-4ad6-b275-2a395b0e0dcb None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:29.937152 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:29.942627 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f98c628-dab7-4496-9a6d-20b354bcf8d4 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:29.984718 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51596395-3fdb-492b-924a-4c6c58ae511e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:29.985408 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51596395-3fdb-492b-924a-4c6c58ae511e None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:29.986393 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51596395-3fdb-492b-924a-4c6c58ae511e None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:29.986685 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51596395-3fdb-492b-924a-4c6c58ae511e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:30.001857 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51596395-3fdb-492b-924a-4c6c58ae511e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:22:30.006806 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:30.009900 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-51596395-3fdb-492b-924a-4c6c58ae511e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:22:30.009900 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:30.009900 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:30.009900 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:30.009900 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:30.009900 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:30.009900 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:30.009900 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:30.009900 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:30.009900 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:30.009900 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:30.009900 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:30.009900 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:22:30.009900 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:22:30.009900 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:22:30.009900 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:22:30.011051 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:22:30.011051 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:22:30.011051 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:22:30.011051 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:22:30.011051 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:30.011051 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:30.011051 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:30.011051 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:30.011051 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:30.011051 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:30.011051 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:30.011051 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:30.011051 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:30.011051 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:30.011051 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:30.011051 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:30.012136 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:30.012136 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:30.012136 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:30.012136 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:30.012136 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:22:30.012136 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:22:30.012136 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:22:30.012136 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:22:30.012136 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:22:30.012136 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:22:30.012136 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:22:30.012136 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:30.014691 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 400/793] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Aug 24 06:22:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:22:30.017682 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51596395-3fdb-492b-924a-4c6c58ae511e None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:30.039312 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf41dcc-a39a-4ad6-b275-2a395b0e0dcb None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:30.079109 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9be756-a656-4d16-8fda-ced5c0dde664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:30.079904 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9be756-a656-4d16-8fda-ced5c0dde664 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:30.080296 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9be756-a656-4d16-8fda-ced5c0dde664 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:30.080683 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9be756-a656-4d16-8fda-ced5c0dde664 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:30.093089 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9be756-a656-4d16-8fda-ced5c0dde664 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:30.094612 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9be756-a656-4d16-8fda-ced5c0dde664 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:30.111087 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-8a9be756-a656-4d16-8fda-ced5c0dde664 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:30.111087 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:30.111087 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:30.111087 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:30.111087 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:30.111087 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:30.111087 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:30.111087 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:30.111087 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:30.111087 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:30.111087 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:30.111087 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:30.111087 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:22:30.111087 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:22:30.111087 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:22:30.111087 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:22:30.112317 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:30.112317 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:30.112317 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:30.112317 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:30.112317 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:30.112317 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:30.112317 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:30.112317 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:30.112317 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:30.112317 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:30.112317 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:30.112317 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:30.112317 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:30.112317 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:30.112317 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:30.112317 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:30.112317 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:22:30.113450 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:22:30.113450 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:30.113450 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:30.119293 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 394/794] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:22:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:22:30.119293 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9be756-a656-4d16-8fda-ced5c0dde664 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:30.167450 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51596395-3fdb-492b-924a-4c6c58ae511e None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:30.210978 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013ee8ba-b9fe-42da-bbcb-464e9b5393d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:30.212850 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013ee8ba-b9fe-42da-bbcb-464e9b5393d2 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:30.213222 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013ee8ba-b9fe-42da-bbcb-464e9b5393d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:30.213576 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013ee8ba-b9fe-42da-bbcb-464e9b5393d2 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:30.214519 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013ee8ba-b9fe-42da-bbcb-464e9b5393d2 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:30.216145 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013ee8ba-b9fe-42da-bbcb-464e9b5393d2 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:30.230775 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-013ee8ba-b9fe-42da-bbcb-464e9b5393d2 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 06:22:30.230775 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:30.230775 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 06:22:30.230775 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 06:22:30.230775 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 06:22:30.230775 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 06:22:30.230775 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 06:22:30.230775 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:22:30.230775 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 06:22:30.230775 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:30.230775 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 06:22:30.230775 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:30.230775 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:30.230775 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:30.230775 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:30.230775 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:30.230775 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:30.231999 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:30.231999 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:30.231999 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:30.231999 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:30.231999 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:30.231999 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:30.231999 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 06:22:30.231999 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 06:22:30.231999 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 06:22:30.231999 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 06:22:30.231999 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:30.231999 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:30.231999 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:30.231999 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:30.231999 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:30.231999 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:30.232898 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:30.232898 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:30.232898 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:30.232898 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:30.232898 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:30.232898 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:30.232898 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:30.232898 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:30.232898 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:30.232898 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:30.232898 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 06:22:30.232898 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:22:30.232898 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 06:22:30.232898 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:30.232898 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 401/795] 10.222.0.47 () {56 vars in 1129 bytes} [Wed Aug 24 06:22:30 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:30.233731 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013ee8ba-b9fe-42da-bbcb-464e9b5393d2 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:30.267493 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9be756-a656-4d16-8fda-ced5c0dde664 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:30.272656 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:30.312660 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a120f63-e17e-4b94-a366-ff6795c52a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:30.313565 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a120f63-e17e-4b94-a366-ff6795c52a03 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:30.313945 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a120f63-e17e-4b94-a366-ff6795c52a03 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:30.314405 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a120f63-e17e-4b94-a366-ff6795c52a03 None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:30.315276 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a120f63-e17e-4b94-a366-ff6795c52a03 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:30.315610 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a120f63-e17e-4b94-a366-ff6795c52a03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:30.317155 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a120f63-e17e-4b94-a366-ff6795c52a03 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:30.320438 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:30.338867 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 395/796] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 24 06:22:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:30.340070 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a120f63-e17e-4b94-a366-ff6795c52a03 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:30.374706 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013ee8ba-b9fe-42da-bbcb-464e9b5393d2 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:30.429334 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4e8e0b-2b96-45c1-8bce-4dc4093bad1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:30.431822 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4e8e0b-2b96-45c1-8bce-4dc4093bad1c None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:30.432286 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4e8e0b-2b96-45c1-8bce-4dc4093bad1c None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:30.432701 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4e8e0b-2b96-45c1-8bce-4dc4093bad1c None admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:30.433546 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4e8e0b-2b96-45c1-8bce-4dc4093bad1c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:30.434004 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4e8e0b-2b96-45c1-8bce-4dc4093bad1c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:30.435526 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4e8e0b-2b96-45c1-8bce-4dc4093bad1c None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:30.458583 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 402/797] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 24 06:22:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:30.458583 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4e8e0b-2b96-45c1-8bce-4dc4093bad1c None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:30.478602 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a120f63-e17e-4b94-a366-ff6795c52a03 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:30.515534 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d23c36c-4eec-402b-9ed6-99caa8859689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:30.516150 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d23c36c-4eec-402b-9ed6-99caa8859689 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:30.516434 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d23c36c-4eec-402b-9ed6-99caa8859689 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:30.516700 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d23c36c-4eec-402b-9ed6-99caa8859689 None admin] PATH_INFO: `/v3/services` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:30.517418 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d23c36c-4eec-402b-9ed6-99caa8859689 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:30.517682 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d23c36c-4eec-402b-9ed6-99caa8859689 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:30.519065 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d23c36c-4eec-402b-9ed6-99caa8859689 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:30.524433 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 396/798] 10.222.0.47 () {56 vars in 1124 bytes} [Wed Aug 24 06:22:30 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:30.525304 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d23c36c-4eec-402b-9ed6-99caa8859689 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:30.570053 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:30.588577 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4e8e0b-2b96-45c1-8bce-4dc4093bad1c None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:30.593569 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:30.631455 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f5347c-b506-4d4a-a3d6-2114d6f6845a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:30.632109 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f5347c-b506-4d4a-a3d6-2114d6f6845a None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:30.632391 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f5347c-b506-4d4a-a3d6-2114d6f6845a None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:30.632664 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f5347c-b506-4d4a-a3d6-2114d6f6845a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:30.635797 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d23c36c-4eec-402b-9ed6-99caa8859689 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:30.645345 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f5347c-b506-4d4a-a3d6-2114d6f6845a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:30.646803 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f5347c-b506-4d4a-a3d6-2114d6f6845a None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:30.662811 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-70f5347c-b506-4d4a-a3d6-2114d6f6845a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:30.662811 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:30.662811 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:30.662811 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:30.662811 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:30.662811 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:30.662811 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:30.662811 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:30.662811 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:30.662811 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:30.662811 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:30.662811 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:30.662811 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:22:30.662811 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:22:30.662811 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:22:30.662811 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:22:30.664051 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:30.664051 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:30.664051 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:30.664051 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:30.664051 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:30.664051 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:30.664051 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:30.664051 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:30.664051 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:30.664051 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:30.664051 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:30.664051 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:30.664051 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:30.664051 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:30.664051 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:30.664051 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:30.664051 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:22:30.665723 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:22:30.665723 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:30.665723 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:30.665723 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 403/799] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:22:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:30.666038 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2bc200-4b1a-461a-8c05-1f428d4a50a3 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:30.666107 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2bc200-4b1a-461a-8c05-1f428d4a50a3 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:30.666348 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2bc200-4b1a-461a-8c05-1f428d4a50a3 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:30.667561 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 404/800] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:22:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:22:30.668580 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0224721c-63cf-426f-8e4a-efd0df251c67 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:30.668853 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0224721c-63cf-426f-8e4a-efd0df251c67 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:30.669119 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0224721c-63cf-426f-8e4a-efd0df251c67 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:30.670527 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 405/801] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:22:30 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:22:30.671099 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0224721c-63cf-426f-8e4a-efd0df251c67 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:30.684410 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dab396-311b-49ca-8fce-457682e4d1c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:30.685466 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dab396-311b-49ca-8fce-457682e4d1c3 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:30.685921 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dab396-311b-49ca-8fce-457682e4d1c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:30.686352 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dab396-311b-49ca-8fce-457682e4d1c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:30.694946 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dab396-311b-49ca-8fce-457682e4d1c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:30.697358 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dab396-311b-49ca-8fce-457682e4d1c3 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:30.709096 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-c5dab396-311b-49ca-8fce-457682e4d1c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:22:30.709096 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:30.709096 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:30.709096 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:30.709096 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:30.709096 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:30.709096 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:30.709096 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:30.709096 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:30.709096 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:30.709096 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:30.709096 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:30.709096 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 06:22:30.709096 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 06:22:30.709096 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 06:22:30.710129 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 06:22:30.710129 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:30.710129 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:30.710129 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:30.710129 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:30.710129 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:30.710129 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:30.710129 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:30.710129 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:30.710129 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:30.710129 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:30.710129 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:30.710129 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:30.710129 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:30.710129 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:30.710129 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:30.710129 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:30.711033 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 06:22:30.711033 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 06:22:30.711033 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:22:30.711033 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:22:30.711033 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:22:30.711033 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:22:30.711033 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:22:30.711033 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:30.711033 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 397/802] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 24 06:22:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:30.711555 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dab396-311b-49ca-8fce-457682e4d1c3 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:30.781086 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0224721c-63cf-426f-8e4a-efd0df251c67 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:30.819996 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e37c75-71cf-4631-bb39-8295112e33cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:30.820600 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e37c75-71cf-4631-bb39-8295112e33cb None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:30.821416 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e37c75-71cf-4631-bb39-8295112e33cb None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:30.821583 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e37c75-71cf-4631-bb39-8295112e33cb None admin] PATH_INFO: `/v3/services` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:30.822427 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e37c75-71cf-4631-bb39-8295112e33cb None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:30.822769 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e37c75-71cf-4631-bb39-8295112e33cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:30.823305 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dab396-311b-49ca-8fce-457682e4d1c3 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:30.824202 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e37c75-71cf-4631-bb39-8295112e33cb None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:30.825761 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:30.830217 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 406/803] 10.222.0.47 () {56 vars in 1124 bytes} [Wed Aug 24 06:22:30 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:30.831453 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851e9aab-a321-4dc1-b341-8d44d574c5b2 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:30.831755 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851e9aab-a321-4dc1-b341-8d44d574c5b2 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:30.832004 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851e9aab-a321-4dc1-b341-8d44d574c5b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:30.847028 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:30.849988 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-851e9aab-a321-4dc1-b341-8d44d574c5b2 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:22:30.851196 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-851e9aab-a321-4dc1-b341-8d44d574c5b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:22:30.861700 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874799ec-ca44-4a4b-9c5f-4fd53ac32c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:30.862313 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874799ec-ca44-4a4b-9c5f-4fd53ac32c4a None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:30.862963 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874799ec-ca44-4a4b-9c5f-4fd53ac32c4a None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:30.862963 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874799ec-ca44-4a4b-9c5f-4fd53ac32c4a None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:30.863502 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874799ec-ca44-4a4b-9c5f-4fd53ac32c4a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:30.866721 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874799ec-ca44-4a4b-9c5f-4fd53ac32c4a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:30.866721 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874799ec-ca44-4a4b-9c5f-4fd53ac32c4a None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:30.883814 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 398/804] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 24 06:22:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:30.883814 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ec18b-d458-4a06-bccf-3e57e75c3141 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:30.883814 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ec18b-d458-4a06-bccf-3e57e75c3141 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:30.883814 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ec18b-d458-4a06-bccf-3e57e75c3141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:30.900554 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-5f8ec18b-d458-4a06-bccf-3e57e75c3141 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:22:30.901781 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-5f8ec18b-d458-4a06-bccf-3e57e75c3141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:22:30.925912 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 407/805] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:22:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:22:30.926763 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851e9aab-a321-4dc1-b341-8d44d574c5b2 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:30.972819 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 399/806] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:22:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:22:30.973671 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8ec18b-d458-4a06-bccf-3e57e75c3141 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:31.056399 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851e9aab-a321-4dc1-b341-8d44d574c5b2 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:31.085074 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:31.094300 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377fa5f6-797f-43df-87d1-0e8fc22e06b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:31.095026 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377fa5f6-797f-43df-87d1-0e8fc22e06b6 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:31.095315 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377fa5f6-797f-43df-87d1-0e8fc22e06b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:31.095592 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377fa5f6-797f-43df-87d1-0e8fc22e06b6 None admin] PATH_INFO: `/v3/projects` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:31.096319 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377fa5f6-797f-43df-87d1-0e8fc22e06b6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:31.096778 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377fa5f6-797f-43df-87d1-0e8fc22e06b6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:31.098630 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377fa5f6-797f-43df-87d1-0e8fc22e06b6 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:31.115571 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8ec18b-d458-4a06-bccf-3e57e75c3141 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:31.120012 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 408/807] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Aug 24 06:22:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:31.121666 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377fa5f6-797f-43df-87d1-0e8fc22e06b6 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:31.132830 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:31.160180 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e238f3e-8e7b-452b-96e4-89d5c08baf01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:31.161157 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e238f3e-8e7b-452b-96e4-89d5c08baf01 None admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:31.161760 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e238f3e-8e7b-452b-96e4-89d5c08baf01 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:31.162240 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e238f3e-8e7b-452b-96e4-89d5c08baf01 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:31.163104 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e238f3e-8e7b-452b-96e4-89d5c08baf01 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:31.165071 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e238f3e-8e7b-452b-96e4-89d5c08baf01 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:31.237360 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377fa5f6-797f-43df-87d1-0e8fc22e06b6 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:31.271306 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d8ee88-66d2-4670-9dc8-93d144bc1685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:31.272241 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d8ee88-66d2-4670-9dc8-93d144bc1685 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:31.272517 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d8ee88-66d2-4670-9dc8-93d144bc1685 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:31.272782 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d8ee88-66d2-4670-9dc8-93d144bc1685 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:31.280556 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d8ee88-66d2-4670-9dc8-93d144bc1685 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:31.282025 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d8ee88-66d2-4670-9dc8-93d144bc1685 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:31.294740 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-70d8ee88-66d2-4670-9dc8-93d144bc1685 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 06:22:31.294740 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:31.294740 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:31.294740 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:31.294740 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:31.294740 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:31.294740 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:31.294740 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:31.294740 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:31.294740 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:31.294740 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:31.294740 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:31.294740 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 06:22:31.294740 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 06:22:31.294740 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 06:22:31.294740 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 06:22:31.296665 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:31.296665 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:31.296665 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 06:22:31.296665 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:22:31.296665 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 06:22:31.296665 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:22:31.296665 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:31.296665 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:31.296665 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:31.296665 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:31.296665 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:31.296665 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:31.296665 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:31.296665 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:31.296665 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:31.296665 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:31.296665 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:31.296665 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:31.298865 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:31.298865 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:31.298865 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 06:22:31.298865 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 06:22:31.298865 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 06:22:31.298865 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 06:22:31.298865 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 06:22:31.298865 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 06:22:31.298865 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 06:22:31.298865 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:31.298865 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 409/808] 10.222.0.47 () {56 vars in 1093 bytes} [Wed Aug 24 06:22:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:31.298865 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d8ee88-66d2-4670-9dc8-93d144bc1685 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:31.316198 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 400/809] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:22:30 2022] POST /identity/v3/endpoints => generated 346 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:22:31.316198 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e238f3e-8e7b-452b-96e4-89d5c08baf01 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:31.365999 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:31.437872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d8ee88-66d2-4670-9dc8-93d144bc1685 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:31.448154 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e238f3e-8e7b-452b-96e4-89d5c08baf01 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:31.480650 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaa1cf5-8935-4b2e-8572-475a503c5a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:31.481514 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaa1cf5-8935-4b2e-8572-475a503c5a28 None admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:31.481797 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaa1cf5-8935-4b2e-8572-475a503c5a28 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:31.482077 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaa1cf5-8935-4b2e-8572-475a503c5a28 None admin] PATH_INFO: `/v3/projects/c02e49a89cdd40d885bfa816f06c58a1/users/02f1cbaf2cd6427585f298c9d58c319a/roles/486144ad1bcd417e86acdbc1d14327bf` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:31.485262 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaa1cf5-8935-4b2e-8572-475a503c5a28 None admin] RBAC: Authorizing `identity:create_grant(project_id=c02e49a89cdd40d885bfa816f06c58a1, user_id=02f1cbaf2cd6427585f298c9d58c319a, role_id=486144ad1bcd417e86acdbc1d14327bf)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:31.486786 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaa1cf5-8935-4b2e-8572-475a503c5a28 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:31.497728 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1099e9-4391-4a44-a060-6818a8f7a5c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:31.498382 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1099e9-4391-4a44-a060-6818a8f7a5c7 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:31.498685 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1099e9-4391-4a44-a060-6818a8f7a5c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:31.498954 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1099e9-4391-4a44-a060-6818a8f7a5c7 None admin] PATH_INFO: `/v3/services/network` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:31.499556 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1099e9-4391-4a44-a060-6818a8f7a5c7 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:31.500964 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1099e9-4391-4a44-a060-6818a8f7a5c7 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:31.510879 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-3c1099e9-4391-4a44-a060-6818a8f7a5c7 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 06:22:31.510879 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:31.510879 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 06:22:31.510879 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 06:22:31.510879 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 06:22:31.510879 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 06:22:31.510879 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 06:22:31.510879 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:22:31.510879 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 06:22:31.510879 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:31.510879 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 06:22:31.510879 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:31.510879 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:31.510879 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:31.510879 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:31.510879 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:31.510879 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:31.512122 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:31.512122 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:31.512122 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:31.512122 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:31.512122 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:31.512122 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:31.512122 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 06:22:31.512122 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 06:22:31.512122 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 06:22:31.512122 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 06:22:31.512122 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:31.512122 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:31.512122 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:31.512122 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:31.512122 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:31.512122 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:31.512958 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:31.512958 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:31.512958 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:31.512958 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:31.512958 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:31.512958 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:31.512958 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:31.512958 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:31.512958 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:31.512958 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:31.512958 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 06:22:31.512958 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:22:31.512958 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 06:22:31.512958 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:31.512958 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 401/810] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 24 06:22:31 2022] GET /identity/v3/services/network => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:31.513917 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1099e9-4391-4a44-a060-6818a8f7a5c7 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:31.546853 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:31.597306 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 410/811] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Aug 24 06:22:31 2022] PUT /identity/v3/projects/c02e49a89cdd40d885bfa816f06c58a1/users/02f1cbaf2cd6427585f298c9d58c319a/roles/486144ad1bcd417e86acdbc1d14327bf => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:22:31.597306 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaa1cf5-8935-4b2e-8572-475a503c5a28 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:31.639120 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1099e9-4391-4a44-a060-6818a8f7a5c7 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:31.686258 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d2d2da-8c17-45a0-b882-c912b3889d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:31.687299 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d2d2da-8c17-45a0-b882-c912b3889d38 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:31.687794 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d2d2da-8c17-45a0-b882-c912b3889d38 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:31.688310 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d2d2da-8c17-45a0-b882-c912b3889d38 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:31.703482 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d2d2da-8c17-45a0-b882-c912b3889d38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:22:31.708900 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-69d2d2da-8c17-45a0-b882-c912b3889d38 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:22:31.708900 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:31.708900 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:31.708900 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:31.708900 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:31.708900 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:31.708900 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:31.708900 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:31.708900 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:31.708900 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:31.708900 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:31.708900 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:31.708900 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:22:31.708900 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:22:31.708900 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:22:31.708900 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:22:31.709975 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:22:31.709975 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:22:31.709975 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:22:31.709975 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:22:31.709975 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:31.709975 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:31.709975 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:31.709975 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:31.709975 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:31.709975 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:31.709975 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:31.709975 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:31.709975 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:31.709975 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:31.709975 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:31.709975 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:31.710804 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:31.710804 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:31.710804 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:31.710804 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:31.710804 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:22:31.710804 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:22:31.710804 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:22:31.710804 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:22:31.710804 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:22:31.710804 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:22:31.710804 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:22:31.710804 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:31.719406 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 402/812] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Aug 24 06:22:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:31.720711 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d2d2da-8c17-45a0-b882-c912b3889d38 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:31.772444 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaa1cf5-8935-4b2e-8572-475a503c5a28 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:31.785949 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:31.823270 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6095c58e-845a-48b3-a7c9-a1ac7a697a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:31.824414 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6095c58e-845a-48b3-a7c9-a1ac7a697a8d None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:31.824752 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6095c58e-845a-48b3-a7c9-a1ac7a697a8d None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:31.825093 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6095c58e-845a-48b3-a7c9-a1ac7a697a8d None admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:31.825935 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6095c58e-845a-48b3-a7c9-a1ac7a697a8d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:31.826305 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6095c58e-845a-48b3-a7c9-a1ac7a697a8d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:31.827821 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6095c58e-845a-48b3-a7c9-a1ac7a697a8d None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:31.845688 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:31.847393 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 411/813] 10.222.0.47 () {56 vars in 1133 bytes} [Wed Aug 24 06:22:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:31.848244 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6095c58e-845a-48b3-a7c9-a1ac7a697a8d None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:31.871634 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d2d2da-8c17-45a0-b882-c912b3889d38 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:31.923933 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fbdf1d-09e3-4c73-b08f-a6b4ae79ad9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:31.924551 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fbdf1d-09e3-4c73-b08f-a6b4ae79ad9b None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:31.924831 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fbdf1d-09e3-4c73-b08f-a6b4ae79ad9b None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:31.925162 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fbdf1d-09e3-4c73-b08f-a6b4ae79ad9b None admin] PATH_INFO: `/v3/services` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:31.925926 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fbdf1d-09e3-4c73-b08f-a6b4ae79ad9b None admin] RBAC: Adding query filter params (name=network) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:31.926234 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fbdf1d-09e3-4c73-b08f-a6b4ae79ad9b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:31.927716 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fbdf1d-09e3-4c73-b08f-a6b4ae79ad9b None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:31.934800 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 403/814] 10.222.0.47 () {56 vars in 1112 bytes} [Wed Aug 24 06:22:31 2022] GET /identity/v3/services?name=network => generated 124 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:31.935868 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fbdf1d-09e3-4c73-b08f-a6b4ae79ad9b None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:31.973662 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6095c58e-845a-48b3-a7c9-a1ac7a697a8d None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:32.013589 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a710f414-4e8b-4d36-af94-daa275a5e980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:32.014373 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a710f414-4e8b-4d36-af94-daa275a5e980 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:32.014724 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a710f414-4e8b-4d36-af94-daa275a5e980 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:32.015067 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a710f414-4e8b-4d36-af94-daa275a5e980 None admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:32.015870 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a710f414-4e8b-4d36-af94-daa275a5e980 None admin] RBAC: Adding query filter params (name=service) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:32.016327 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a710f414-4e8b-4d36-af94-daa275a5e980 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:32.017958 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a710f414-4e8b-4d36-af94-daa275a5e980 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:32.032682 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 412/815] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Aug 24 06:22:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:32.033923 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a710f414-4e8b-4d36-af94-daa275a5e980 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:32.052509 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fbdf1d-09e3-4c73-b08f-a6b4ae79ad9b None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:32.066672 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:32.093332 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdffb89-228e-472f-b92a-7829da56e67f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:32.094249 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdffb89-228e-472f-b92a-7829da56e67f None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:32.094679 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdffb89-228e-472f-b92a-7829da56e67f None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:32.095095 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdffb89-228e-472f-b92a-7829da56e67f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:32.119537 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:32.148284 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a710f414-4e8b-4d36-af94-daa275a5e980 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:32.177502 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdffb89-228e-472f-b92a-7829da56e67f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:32.179073 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdffb89-228e-472f-b92a-7829da56e67f None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:32.188529 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e054e20-5acc-4367-aabc-ad98e60a7027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:32.189418 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e054e20-5acc-4367-aabc-ad98e60a7027 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:32.189850 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e054e20-5acc-4367-aabc-ad98e60a7027 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:32.190300 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e054e20-5acc-4367-aabc-ad98e60a7027 None admin] PATH_INFO: `/v3/services` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:32.191196 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e054e20-5acc-4367-aabc-ad98e60a7027 None admin] RBAC: Adding query filter params (type=network) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:32.191645 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e054e20-5acc-4367-aabc-ad98e60a7027 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:32.193312 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e054e20-5acc-4367-aabc-ad98e60a7027 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:32.199660 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 413/816] 10.222.0.47 () {56 vars in 1112 bytes} [Wed Aug 24 06:22:32 2022] GET /identity/v3/services?type=network => generated 350 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:32.200706 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e054e20-5acc-4367-aabc-ad98e60a7027 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:32.300794 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 404/817] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 24 06:22:31 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 06:22:32.301734 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdffb89-228e-472f-b92a-7829da56e67f None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:32.308323 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:32.317832 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e054e20-5acc-4367-aabc-ad98e60a7027 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:32.357847 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8572215c-8f79-4909-9c66-8dfeca8cb926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:32.358486 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8572215c-8f79-4909-9c66-8dfeca8cb926 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:32.358904 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8572215c-8f79-4909-9c66-8dfeca8cb926 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:32.359121 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8572215c-8f79-4909-9c66-8dfeca8cb926 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:32.360413 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:32.378412 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8572215c-8f79-4909-9c66-8dfeca8cb926 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:32.378412 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8572215c-8f79-4909-9c66-8dfeca8cb926 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:32.390817 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-8572215c-8f79-4909-9c66-8dfeca8cb926 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:32.390817 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:32.390817 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:32.390817 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:32.390817 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:32.390817 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:32.390817 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:32.390817 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:32.390817 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:32.390817 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:32.390817 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:32.390817 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:32.390817 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:22:32.390817 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:22:32.390817 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:22:32.390817 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:22:32.391979 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:32.391979 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:32.391979 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:32.391979 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:32.391979 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:32.391979 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:32.391979 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:32.391979 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:32.391979 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:32.391979 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:32.391979 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:32.391979 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:32.391979 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:32.391979 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:32.391979 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:32.391979 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:32.391979 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:22:32.393067 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:22:32.393067 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:32.393067 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:32.400250 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 414/818] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:22:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:32.401443 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8572215c-8f79-4909-9c66-8dfeca8cb926 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:32.429335 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdffb89-228e-472f-b92a-7829da56e67f None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:32.474656 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546c8b7e-a16b-4fe5-9170-96dd54ea8404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:32.475388 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546c8b7e-a16b-4fe5-9170-96dd54ea8404 None admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:32.475678 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546c8b7e-a16b-4fe5-9170-96dd54ea8404 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:32.475956 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546c8b7e-a16b-4fe5-9170-96dd54ea8404 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:32.477063 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546c8b7e-a16b-4fe5-9170-96dd54ea8404 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:32.478210 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546c8b7e-a16b-4fe5-9170-96dd54ea8404 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:32.514181 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8572215c-8f79-4909-9c66-8dfeca8cb926 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:32.557053 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 405/819] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:22:32 2022] POST /identity/v3/endpoints => generated 337 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:22:32.565213 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546c8b7e-a16b-4fe5-9170-96dd54ea8404 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:32.586304 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5578280f-15b5-43f7-9049-fb73657d6b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:32.587282 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5578280f-15b5-43f7-9049-fb73657d6b0c None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:32.587699 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5578280f-15b5-43f7-9049-fb73657d6b0c None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:32.587976 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5578280f-15b5-43f7-9049-fb73657d6b0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:32.612123 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:32.621306 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5578280f-15b5-43f7-9049-fb73657d6b0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:32.621306 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5578280f-15b5-43f7-9049-fb73657d6b0c None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:32.658893 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-5578280f-15b5-43f7-9049-fb73657d6b0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:32.658893 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:32.658893 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:32.658893 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:32.658893 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:32.658893 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:32.658893 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:32.658893 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:32.658893 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:32.658893 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:32.658893 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:32.658893 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:32.658893 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:22:32.658893 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:22:32.658893 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:22:32.658893 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:22:32.659880 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:32.659880 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:32.659880 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:32.659880 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:32.659880 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:32.659880 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:32.659880 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:32.659880 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:32.659880 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:32.659880 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:32.659880 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:32.659880 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:32.659880 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:32.659880 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:32.659880 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:32.659880 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:32.659880 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:22:32.660759 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:22:32.660759 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:32.660759 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:32.669675 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 415/820] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:22:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:32.682284 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5578280f-15b5-43f7-9049-fb73657d6b0c None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:32.779939 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546c8b7e-a16b-4fe5-9170-96dd54ea8404 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:32.790577 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:32.827935 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7486806-1f72-472e-b2b2-9752880f2d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:32.829350 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7486806-1f72-472e-b2b2-9752880f2d8e None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:32.830621 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7486806-1f72-472e-b2b2-9752880f2d8e None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:32.831051 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7486806-1f72-472e-b2b2-9752880f2d8e None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:32.832786 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7486806-1f72-472e-b2b2-9752880f2d8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:32.833460 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7486806-1f72-472e-b2b2-9752880f2d8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:32.835135 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7486806-1f72-472e-b2b2-9752880f2d8e None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:32.844373 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5578280f-15b5-43f7-9049-fb73657d6b0c None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:32.849737 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 406/821] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 24 06:22:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:32.857802 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7486806-1f72-472e-b2b2-9752880f2d8e None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:32.892693 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b26eec-da3b-4a17-8441-d4aae876528e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:32.893440 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b26eec-da3b-4a17-8441-d4aae876528e None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:32.893795 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b26eec-da3b-4a17-8441-d4aae876528e None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:32.894166 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b26eec-da3b-4a17-8441-d4aae876528e None admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:32.895021 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b26eec-da3b-4a17-8441-d4aae876528e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:32.895403 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b26eec-da3b-4a17-8441-d4aae876528e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:32.897206 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b26eec-da3b-4a17-8441-d4aae876528e None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:32.911672 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 416/822] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 24 06:22:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:32.921559 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b26eec-da3b-4a17-8441-d4aae876528e None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:32.988743 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7486806-1f72-472e-b2b2-9752880f2d8e None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:32.991375 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:33.028696 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5badc80-baec-4017-88fd-436a1cb42a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:33.029722 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5badc80-baec-4017-88fd-436a1cb42a30 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:33.030177 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5badc80-baec-4017-88fd-436a1cb42a30 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:33.030636 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5badc80-baec-4017-88fd-436a1cb42a30 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:33.041824 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5badc80-baec-4017-88fd-436a1cb42a30 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:33.043520 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5badc80-baec-4017-88fd-436a1cb42a30 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:33.050842 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b26eec-da3b-4a17-8441-d4aae876528e None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:33.057801 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-c5badc80-baec-4017-88fd-436a1cb42a30 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 06:22:33.057801 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:33.057801 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:33.057801 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:33.057801 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:33.057801 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:33.057801 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:33.057801 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:33.057801 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:33.057801 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:33.057801 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:33.057801 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:33.057801 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 06:22:33.057801 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 06:22:33.057801 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 06:22:33.057801 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 06:22:33.058889 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:33.058889 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:33.058889 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 06:22:33.058889 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:22:33.058889 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 06:22:33.058889 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:22:33.058889 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:33.058889 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:33.058889 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:33.058889 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:33.058889 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:33.058889 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:33.058889 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:33.058889 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:33.058889 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:33.058889 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:33.058889 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:33.058889 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:33.059873 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:33.059873 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:33.059873 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 06:22:33.059873 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 06:22:33.059873 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 06:22:33.059873 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 06:22:33.059873 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 06:22:33.059873 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 06:22:33.059873 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 06:22:33.059873 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:33.062128 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 407/823] 10.222.0.47 () {56 vars in 1117 bytes} [Wed Aug 24 06:22:32 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:33.070322 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5badc80-baec-4017-88fd-436a1cb42a30 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:33.107681 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f900bfbb-ea3c-4b82-9c34-163c80ed2e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:33.108350 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f900bfbb-ea3c-4b82-9c34-163c80ed2e4c None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:33.109468 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f900bfbb-ea3c-4b82-9c34-163c80ed2e4c None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:33.110743 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f900bfbb-ea3c-4b82-9c34-163c80ed2e4c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:33.113790 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:33.121911 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f900bfbb-ea3c-4b82-9c34-163c80ed2e4c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:33.123721 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f900bfbb-ea3c-4b82-9c34-163c80ed2e4c None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:33.139477 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-f900bfbb-ea3c-4b82-9c34-163c80ed2e4c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:22:33.139477 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:33.139477 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:33.139477 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:33.139477 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:33.139477 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:33.139477 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:33.139477 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:33.139477 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:33.139477 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:33.139477 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:33.139477 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:33.139477 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 06:22:33.139477 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 06:22:33.139477 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 06:22:33.140446 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 06:22:33.140446 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:33.140446 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:33.140446 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:33.140446 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:33.140446 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:33.140446 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:33.140446 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:33.140446 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:33.140446 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:33.140446 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:33.140446 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:33.140446 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:33.140446 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:33.140446 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:33.140446 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:33.140446 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:33.141365 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 06:22:33.141365 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 06:22:33.141365 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:22:33.141365 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:22:33.141365 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:22:33.141365 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:22:33.141365 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:22:33.141365 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:33.143467 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 417/824] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 24 06:22:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:22:33.151477 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f900bfbb-ea3c-4b82-9c34-163c80ed2e4c None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:33.221780 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5badc80-baec-4017-88fd-436a1cb42a30 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:33.262667 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cb9ed3-8123-49c9-9bb6-0676c444f1be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:33.263701 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cb9ed3-8123-49c9-9bb6-0676c444f1be None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:33.264216 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cb9ed3-8123-49c9-9bb6-0676c444f1be None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:33.264686 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cb9ed3-8123-49c9-9bb6-0676c444f1be None admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:33.265698 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cb9ed3-8123-49c9-9bb6-0676c444f1be None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:33.265876 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f900bfbb-ea3c-4b82-9c34-163c80ed2e4c None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:33.266372 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cb9ed3-8123-49c9-9bb6-0676c444f1be None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:33.268202 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cb9ed3-8123-49c9-9bb6-0676c444f1be None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:33.280801 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:33.290739 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 408/825] 10.222.0.47 () {56 vars in 1149 bytes} [Wed Aug 24 06:22:33 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:33.297887 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cb9ed3-8123-49c9-9bb6-0676c444f1be None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:33.311980 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f2d282-fd3f-491c-ba91-39af4d004a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:33.313392 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f2d282-fd3f-491c-ba91-39af4d004a64 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:33.313880 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f2d282-fd3f-491c-ba91-39af4d004a64 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:33.314694 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f2d282-fd3f-491c-ba91-39af4d004a64 None admin] PATH_INFO: `/v3/projects` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:33.315909 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f2d282-fd3f-491c-ba91-39af4d004a64 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:33.316435 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f2d282-fd3f-491c-ba91-39af4d004a64 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:33.318994 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f2d282-fd3f-491c-ba91-39af4d004a64 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:33.341866 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 418/826] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Aug 24 06:22:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:33.343392 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f2d282-fd3f-491c-ba91-39af4d004a64 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:33.423052 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:33.443466 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cb9ed3-8123-49c9-9bb6-0676c444f1be None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:33.487469 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f2d282-fd3f-491c-ba91-39af4d004a64 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:33.489225 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7864ed-9944-4ada-bb47-8a504df24d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:33.489855 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7864ed-9944-4ada-bb47-8a504df24d34 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:33.490182 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7864ed-9944-4ada-bb47-8a504df24d34 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:33.490510 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7864ed-9944-4ada-bb47-8a504df24d34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:33.501899 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7864ed-9944-4ada-bb47-8a504df24d34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:33.503424 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7864ed-9944-4ada-bb47-8a504df24d34 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:33.525693 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-7a7864ed-9944-4ada-bb47-8a504df24d34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:33.525693 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:33.525693 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:33.525693 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:33.525693 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:33.525693 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:33.525693 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:33.525693 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:33.525693 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:33.525693 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:33.525693 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:33.525693 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:33.525693 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:22:33.525693 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:22:33.525693 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:22:33.525693 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:22:33.526707 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:33.526707 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:33.526707 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:33.526707 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:33.526707 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:33.526707 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:33.526707 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:33.526707 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:33.526707 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:33.526707 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:33.526707 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:33.526707 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:33.526707 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:33.526707 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:33.526707 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:33.526707 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:33.526707 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:22:33.527623 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:22:33.527623 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:33.527623 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:33.529590 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef7d7fa-c5ce-44b3-9cab-1ce7e1dbf0ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:33.530213 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef7d7fa-c5ce-44b3-9cab-1ce7e1dbf0ee None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:33.530496 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef7d7fa-c5ce-44b3-9cab-1ce7e1dbf0ee None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:33.530827 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef7d7fa-c5ce-44b3-9cab-1ce7e1dbf0ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:33.531568 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 409/827] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:22:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:33.531673 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef7d7fa-c5ce-44b3-9cab-1ce7e1dbf0ee None admin] RBAC: Adding query filter params (role.id=486144ad1bcd417e86acdbc1d14327bf, scope.project.id=c02e49a89cdd40d885bfa816f06c58a1, user.id=1f4f49c1b08c460f8ba6c451c9c92c2a) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:33.531910 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef7d7fa-c5ce-44b3-9cab-1ce7e1dbf0ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:33.532700 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1c9696-f5f6-49e6-8627-39ee74c1762a None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:33.532986 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1c9696-f5f6-49e6-8627-39ee74c1762a None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:33.533287 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1c9696-f5f6-49e6-8627-39ee74c1762a None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:33.533442 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef7d7fa-c5ce-44b3-9cab-1ce7e1dbf0ee None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:33.538631 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 410/828] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:22:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:22:33.538631 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1c9696-f5f6-49e6-8627-39ee74c1762a None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:33.546874 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 419/829] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 24 06:22:33 2022] GET /identity/v3/role_assignments?user.id=1f4f49c1b08c460f8ba6c451c9c92c2a&scope.project.id=c02e49a89cdd40d885bfa816f06c58a1&role.id=486144ad1bcd417e86acdbc1d14327bf => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:22:33.554784 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac9ec92-6f46-4796-aca9-c2c903b7af92 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:33.555740 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac9ec92-6f46-4796-aca9-c2c903b7af92 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:33.556032 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac9ec92-6f46-4796-aca9-c2c903b7af92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:33.578146 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-aac9ec92-6f46-4796-aca9-c2c903b7af92 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:22:33.580560 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-aac9ec92-6f46-4796-aca9-c2c903b7af92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:22:33.622116 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:33.681764 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 420/830] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:22:33 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:22:33.688173 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1c9696-f5f6-49e6-8627-39ee74c1762a None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:33.718395 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08036023-d525-43ed-9f66-5810b581745b None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:33.718766 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08036023-d525-43ed-9f66-5810b581745b None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:33.719110 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08036023-d525-43ed-9f66-5810b581745b None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:33.720767 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 421/831] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:22:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 06:22:33.727543 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272b7b6c-91de-4292-ac3a-aa34bf31f8b7 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:33.727897 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272b7b6c-91de-4292-ac3a-aa34bf31f8b7 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:33.728251 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272b7b6c-91de-4292-ac3a-aa34bf31f8b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:33.732799 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0669112a-20a3-4b03-a8c3-f52fb44f6a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:33.733447 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0669112a-20a3-4b03-a8c3-f52fb44f6a72 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:33.733729 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0669112a-20a3-4b03-a8c3-f52fb44f6a72 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:33.734008 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0669112a-20a3-4b03-a8c3-f52fb44f6a72 None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:33.734664 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0669112a-20a3-4b03-a8c3-f52fb44f6a72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:33.734969 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0669112a-20a3-4b03-a8c3-f52fb44f6a72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:33.736341 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0669112a-20a3-4b03-a8c3-f52fb44f6a72 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:33.739473 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:33.748634 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-272b7b6c-91de-4292-ac3a-aa34bf31f8b7 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:22:33.749869 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-272b7b6c-91de-4292-ac3a-aa34bf31f8b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:22:33.756944 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 411/832] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 24 06:22:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:33.757581 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0669112a-20a3-4b03-a8c3-f52fb44f6a72 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:33.833482 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 422/833] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:22:33 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 06:22:33.887532 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0669112a-20a3-4b03-a8c3-f52fb44f6a72 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:33.924414 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc02f029-f285-40bc-9a81-15ae61946d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:33.925145 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc02f029-f285-40bc-9a81-15ae61946d4e None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:33.925446 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc02f029-f285-40bc-9a81-15ae61946d4e None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:33.925718 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc02f029-f285-40bc-9a81-15ae61946d4e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:33.933802 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc02f029-f285-40bc-9a81-15ae61946d4e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:33.935271 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc02f029-f285-40bc-9a81-15ae61946d4e None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:33.947578 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-bc02f029-f285-40bc-9a81-15ae61946d4e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:22:33.947578 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:33.947578 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:33.947578 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:33.947578 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:33.947578 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:33.947578 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:33.947578 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:33.947578 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:33.947578 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:33.947578 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:33.947578 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:33.947578 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 06:22:33.947578 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 06:22:33.947578 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 06:22:33.948520 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 06:22:33.948520 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:33.948520 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:33.948520 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:33.948520 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:33.948520 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:33.948520 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:33.948520 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:33.948520 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:33.948520 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:33.948520 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:33.948520 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:33.948520 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:33.948520 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:33.948520 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:33.948520 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:33.948520 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:33.949451 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 06:22:33.949451 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 06:22:33.949451 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:22:33.949451 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:22:33.949451 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:22:33.949451 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:22:33.949451 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:22:33.949451 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:33.949451 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 412/834] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 24 06:22:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:33.956529 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272b7b6c-91de-4292-ac3a-aa34bf31f8b7 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:33.975113 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc02f029-f285-40bc-9a81-15ae61946d4e None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:34.053788 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:34.078376 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:34.084972 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272b7b6c-91de-4292-ac3a-aa34bf31f8b7 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:34.113752 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc02f029-f285-40bc-9a81-15ae61946d4e None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:34.130148 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd856bb0-3548-456e-94eb-266d540e802c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:34.130860 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd856bb0-3548-456e-94eb-266d540e802c None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:34.131215 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd856bb0-3548-456e-94eb-266d540e802c None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:34.131504 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd856bb0-3548-456e-94eb-266d540e802c None admin] PATH_INFO: `/v3/projects` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:34.132224 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd856bb0-3548-456e-94eb-266d540e802c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:34.132609 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd856bb0-3548-456e-94eb-266d540e802c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:34.134769 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd856bb0-3548-456e-94eb-266d540e802c None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:34.148561 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 423/835] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Aug 24 06:22:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:34.149491 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd856bb0-3548-456e-94eb-266d540e802c None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:34.160675 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fafaf2c-b3ce-49cb-9018-d96a8aa7ee2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:34.161437 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fafaf2c-b3ce-49cb-9018-d96a8aa7ee2b None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:34.161713 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fafaf2c-b3ce-49cb-9018-d96a8aa7ee2b None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:34.161997 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fafaf2c-b3ce-49cb-9018-d96a8aa7ee2b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:34.162576 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fafaf2c-b3ce-49cb-9018-d96a8aa7ee2b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:34.163938 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fafaf2c-b3ce-49cb-9018-d96a8aa7ee2b None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:34.173315 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-5fafaf2c-b3ce-49cb-9018-d96a8aa7ee2b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 06:22:34.173315 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:34.173315 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 06:22:34.173315 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 06:22:34.173315 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 06:22:34.173315 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 06:22:34.173315 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 06:22:34.173315 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:22:34.173315 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 06:22:34.173315 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:34.173315 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 06:22:34.173315 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:34.173315 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:34.173315 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:34.173315 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:34.173315 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:34.173315 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:34.174380 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:34.174380 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:34.174380 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:34.174380 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:34.174380 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:34.174380 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:34.174380 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 06:22:34.174380 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 06:22:34.174380 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 06:22:34.174380 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 06:22:34.174380 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:34.174380 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:34.174380 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:34.174380 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:34.174380 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:34.174380 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:34.178259 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:34.178259 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:34.178259 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:34.178259 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:34.178259 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:34.178259 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:34.178259 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:34.178259 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:34.178259 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:34.178259 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:34.178259 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 06:22:34.178259 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:22:34.178259 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 06:22:34.178259 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:34.178259 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 413/836] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Aug 24 06:22:33 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:34.180485 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fafaf2c-b3ce-49cb-9018-d96a8aa7ee2b None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:34.279249 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd856bb0-3548-456e-94eb-266d540e802c None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:34.299011 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fafaf2c-b3ce-49cb-9018-d96a8aa7ee2b None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:34.315311 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6d2cd2-b6a1-4e82-a2a2-44145c8a56d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:34.315861 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6d2cd2-b6a1-4e82-a2a2-44145c8a56d4 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:34.316136 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6d2cd2-b6a1-4e82-a2a2-44145c8a56d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:34.316403 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6d2cd2-b6a1-4e82-a2a2-44145c8a56d4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:34.323310 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:34.328335 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6d2cd2-b6a1-4e82-a2a2-44145c8a56d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:22:34.333540 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-5c6d2cd2-b6a1-4e82-a2a2-44145c8a56d4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:22:34.333540 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:34.333540 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:34.333540 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:34.333540 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:34.333540 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:34.333540 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:34.333540 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:34.333540 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:34.333540 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:34.333540 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:34.333540 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:34.333540 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:22:34.333540 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:22:34.333540 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:22:34.333540 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:22:34.350335 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:22:34.350335 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:22:34.350335 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:22:34.350335 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:22:34.350335 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:34.350335 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:34.350335 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:34.350335 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:34.350335 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:34.350335 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:34.350335 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:34.350335 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:34.350335 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:34.350335 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:34.350335 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:34.350335 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:34.351203 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:34.351203 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:34.351203 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:34.351203 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:34.351203 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:22:34.351203 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:22:34.351203 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:22:34.351203 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:22:34.351203 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:22:34.351203 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:22:34.351203 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:22:34.351203 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:34.351203 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 424/837] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Aug 24 06:22:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:34.352643 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6e7a7b-ffb5-4a5f-b08c-c160bd49e399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:34.352643 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6e7a7b-ffb5-4a5f-b08c-c160bd49e399 None admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:34.352925 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6d2cd2-b6a1-4e82-a2a2-44145c8a56d4 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:34.353055 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6e7a7b-ffb5-4a5f-b08c-c160bd49e399 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:34.353055 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6e7a7b-ffb5-4a5f-b08c-c160bd49e399 None admin] PATH_INFO: `/v3/projects/c02e49a89cdd40d885bfa816f06c58a1/users/ea70acd744994250a2905b5dd7163dfb/roles/1712d85aedaf4189bf4bc89094d888c3` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:34.361232 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6e7a7b-ffb5-4a5f-b08c-c160bd49e399 None admin] RBAC: Authorizing `identity:create_grant(project_id=c02e49a89cdd40d885bfa816f06c58a1, user_id=ea70acd744994250a2905b5dd7163dfb, role_id=1712d85aedaf4189bf4bc89094d888c3)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:34.362715 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6e7a7b-ffb5-4a5f-b08c-c160bd49e399 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:34.364373 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:34.440918 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 414/838] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Aug 24 06:22:34 2022] PUT /identity/v3/projects/c02e49a89cdd40d885bfa816f06c58a1/users/ea70acd744994250a2905b5dd7163dfb/roles/1712d85aedaf4189bf4bc89094d888c3 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:22:34.441824 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6e7a7b-ffb5-4a5f-b08c-c160bd49e399 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:34.481797 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6d2cd2-b6a1-4e82-a2a2-44145c8a56d4 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:34.523876 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2699f8a7-13f2-4463-b273-08b21541a790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:34.524529 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2699f8a7-13f2-4463-b273-08b21541a790 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:34.525036 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2699f8a7-13f2-4463-b273-08b21541a790 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:34.525354 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2699f8a7-13f2-4463-b273-08b21541a790 None admin] PATH_INFO: `/v3/services` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:34.526043 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2699f8a7-13f2-4463-b273-08b21541a790 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:34.526354 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2699f8a7-13f2-4463-b273-08b21541a790 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:34.527953 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2699f8a7-13f2-4463-b273-08b21541a790 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:34.538713 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 425/839] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Aug 24 06:22:34 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:34.540130 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2699f8a7-13f2-4463-b273-08b21541a790 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:34.586215 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6e7a7b-ffb5-4a5f-b08c-c160bd49e399 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:34.634777 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889af73f-4498-45bd-907d-0f03c067bb39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:34.635427 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889af73f-4498-45bd-907d-0f03c067bb39 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:34.635717 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889af73f-4498-45bd-907d-0f03c067bb39 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:34.636419 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889af73f-4498-45bd-907d-0f03c067bb39 None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:34.637303 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889af73f-4498-45bd-907d-0f03c067bb39 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:34.640719 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889af73f-4498-45bd-907d-0f03c067bb39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:34.640849 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:34.640930 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889af73f-4498-45bd-907d-0f03c067bb39 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:34.657296 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 415/840] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 24 06:22:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:34.660016 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889af73f-4498-45bd-907d-0f03c067bb39 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:34.669333 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2699f8a7-13f2-4463-b273-08b21541a790 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:34.708321 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:34.713009 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d56450-7e19-4598-b3bc-ceb2f49cc7b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:34.713761 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d56450-7e19-4598-b3bc-ceb2f49cc7b2 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:34.714152 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d56450-7e19-4598-b3bc-ceb2f49cc7b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:34.715118 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d56450-7e19-4598-b3bc-ceb2f49cc7b2 None admin] PATH_INFO: `/v3/services` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:34.715244 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d56450-7e19-4598-b3bc-ceb2f49cc7b2 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:34.715742 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d56450-7e19-4598-b3bc-ceb2f49cc7b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:34.717384 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d56450-7e19-4598-b3bc-ceb2f49cc7b2 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:34.723596 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 426/841] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Aug 24 06:22:34 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:34.802255 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889af73f-4498-45bd-907d-0f03c067bb39 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:34.849011 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4596199-360c-41bc-92a6-907e6bf66245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:34.849615 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4596199-360c-41bc-92a6-907e6bf66245 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:34.849891 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4596199-360c-41bc-92a6-907e6bf66245 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:34.850160 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4596199-360c-41bc-92a6-907e6bf66245 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:34.863557 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4596199-360c-41bc-92a6-907e6bf66245 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:34.865154 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4596199-360c-41bc-92a6-907e6bf66245 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:34.881435 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-b4596199-360c-41bc-92a6-907e6bf66245 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:34.881435 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:34.881435 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:34.881435 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:34.881435 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:34.881435 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:34.881435 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:34.881435 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:34.881435 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:34.881435 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:34.881435 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:34.881435 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:34.881435 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:22:34.881435 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:22:34.881435 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:22:34.881435 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:22:34.882537 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:34.882537 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:34.882537 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:34.882537 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:34.882537 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:34.882537 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:34.882537 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:34.882537 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:34.882537 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:34.882537 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:34.882537 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:34.882537 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:34.882537 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:34.882537 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:34.882537 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:34.882537 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:34.882537 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:22:34.884909 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:22:34.884909 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:34.884909 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:34.884909 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 416/842] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:22:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:22:34.891583 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d56450-7e19-4598-b3bc-ceb2f49cc7b2 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:35.020341 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d56450-7e19-4598-b3bc-ceb2f49cc7b2 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:35.060699 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ceee43-37de-4ae9-bbdc-74c55a07e1ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:35.061670 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ceee43-37de-4ae9-bbdc-74c55a07e1ff None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:35.061955 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ceee43-37de-4ae9-bbdc-74c55a07e1ff None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:35.062226 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ceee43-37de-4ae9-bbdc-74c55a07e1ff None admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:35.062874 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ceee43-37de-4ae9-bbdc-74c55a07e1ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:35.063172 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ceee43-37de-4ae9-bbdc-74c55a07e1ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:35.064636 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ceee43-37de-4ae9-bbdc-74c55a07e1ff None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:35.077905 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 427/843] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 24 06:22:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:35.083591 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4596199-360c-41bc-92a6-907e6bf66245 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:35.181743 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:35.198989 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4596199-360c-41bc-92a6-907e6bf66245 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:35.237780 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee233b1-2b61-4d28-b6a0-7e9f30e4929c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:35.238428 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee233b1-2b61-4d28-b6a0-7e9f30e4929c None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:35.238714 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee233b1-2b61-4d28-b6a0-7e9f30e4929c None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:35.238993 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee233b1-2b61-4d28-b6a0-7e9f30e4929c None admin] PATH_INFO: `/v3/users/swift` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:35.248282 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee233b1-2b61-4d28-b6a0-7e9f30e4929c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:35.249778 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee233b1-2b61-4d28-b6a0-7e9f30e4929c None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:35.262870 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-9ee233b1-2b61-4d28-b6a0-7e9f30e4929c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 06:22:35.262870 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:35.262870 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:35.262870 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:35.262870 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:35.262870 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:35.262870 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:35.262870 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:35.262870 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:35.262870 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:35.262870 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:35.262870 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:35.262870 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 06:22:35.262870 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 06:22:35.262870 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 06:22:35.262870 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 06:22:35.263860 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:35.263860 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:35.263860 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 06:22:35.263860 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:22:35.263860 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 06:22:35.263860 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:22:35.263860 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:35.263860 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:35.263860 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:35.263860 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:35.263860 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:35.263860 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:35.263860 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:35.263860 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:35.263860 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:35.263860 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:35.263860 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:35.263860 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:35.264785 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:35.264785 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:35.264785 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 06:22:35.264785 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 06:22:35.264785 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 06:22:35.264785 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 06:22:35.264785 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 06:22:35.264785 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 06:22:35.264785 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 06:22:35.264785 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:35.268799 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 417/844] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Aug 24 06:22:35 2022] GET /identity/v3/users/swift => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:35.274737 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ceee43-37de-4ae9-bbdc-74c55a07e1ff None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:35.279115 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:35.392747 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ceee43-37de-4ae9-bbdc-74c55a07e1ff None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:35.431207 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3894d6e3-f082-46dc-9de3-33ffbf41281b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:35.431893 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3894d6e3-f082-46dc-9de3-33ffbf41281b None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:35.432127 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3894d6e3-f082-46dc-9de3-33ffbf41281b None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:35.432412 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3894d6e3-f082-46dc-9de3-33ffbf41281b None admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:35.433153 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3894d6e3-f082-46dc-9de3-33ffbf41281b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:35.433493 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3894d6e3-f082-46dc-9de3-33ffbf41281b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:35.434950 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3894d6e3-f082-46dc-9de3-33ffbf41281b None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:35.458877 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 428/845] 10.222.0.47 () {56 vars in 1135 bytes} [Wed Aug 24 06:22:35 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:35.465431 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee233b1-2b61-4d28-b6a0-7e9f30e4929c None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:35.579374 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee233b1-2b61-4d28-b6a0-7e9f30e4929c None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:35.616390 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850a5bbb-f6bf-48ec-93aa-5171ca5eb0ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:35.617025 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850a5bbb-f6bf-48ec-93aa-5171ca5eb0ad None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:35.618066 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850a5bbb-f6bf-48ec-93aa-5171ca5eb0ad None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:35.618823 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850a5bbb-f6bf-48ec-93aa-5171ca5eb0ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:35.626170 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8242299-8561-44cd-95e3-4d8e947b8e32 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:35.626571 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8242299-8561-44cd-95e3-4d8e947b8e32 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:35.626734 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8242299-8561-44cd-95e3-4d8e947b8e32 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:35.628307 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 429/846] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:22:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:22:35.634514 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf34fc67-4884-42f8-8f7a-f9227e650c71 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:35.634823 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf34fc67-4884-42f8-8f7a-f9227e650c71 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:35.635065 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf34fc67-4884-42f8-8f7a-f9227e650c71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:35.653063 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:35.668723 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-cf34fc67-4884-42f8-8f7a-f9227e650c71 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:22:35.670462 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-cf34fc67-4884-42f8-8f7a-f9227e650c71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:22:35.701676 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850a5bbb-f6bf-48ec-93aa-5171ca5eb0ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:35.703623 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850a5bbb-f6bf-48ec-93aa-5171ca5eb0ad None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:35.747425 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:35.773424 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 430/847] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:22:35 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 06:22:35.879636 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 418/848] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 24 06:22:35 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 06:22:35.885683 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf34fc67-4884-42f8-8f7a-f9227e650c71 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:36.005279 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf34fc67-4884-42f8-8f7a-f9227e650c71 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:36.041761 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c5cce4-44a2-4757-aed3-6d0a8ecf3af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:36.042456 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c5cce4-44a2-4757-aed3-6d0a8ecf3af7 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:36.042741 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c5cce4-44a2-4757-aed3-6d0a8ecf3af7 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:36.043017 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c5cce4-44a2-4757-aed3-6d0a8ecf3af7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:36.054226 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c5cce4-44a2-4757-aed3-6d0a8ecf3af7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:36.056251 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c5cce4-44a2-4757-aed3-6d0a8ecf3af7 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:36.075083 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-59c5cce4-44a2-4757-aed3-6d0a8ecf3af7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:36.075083 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:36.075083 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:36.075083 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:36.075083 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:36.075083 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:36.075083 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:36.075083 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:36.075083 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:36.075083 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:36.075083 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:36.075083 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:36.075083 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:22:36.075083 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:22:36.075083 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:22:36.075083 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:22:36.076113 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:36.076113 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:36.076113 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:36.076113 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:36.076113 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:36.076113 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:36.076113 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:36.076113 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:36.076113 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:36.076113 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:36.076113 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:36.076113 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:36.076113 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:36.076113 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:36.076113 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:36.076113 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:36.076113 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:22:36.077562 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:22:36.077562 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:36.077562 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:36.077562 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 431/849] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:22:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:36.083689 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850a5bbb-f6bf-48ec-93aa-5171ca5eb0ad None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:36.095613 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c5cce4-44a2-4757-aed3-6d0a8ecf3af7 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:36.145348 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:36.171956 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:36.224070 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850a5bbb-f6bf-48ec-93aa-5171ca5eb0ad None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:36.241804 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c5cce4-44a2-4757-aed3-6d0a8ecf3af7 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:36.267160 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c79b9f2-a612-4dce-bac1-1f1d2c444098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:36.268287 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c79b9f2-a612-4dce-bac1-1f1d2c444098 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:36.269333 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c79b9f2-a612-4dce-bac1-1f1d2c444098 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:36.269862 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c79b9f2-a612-4dce-bac1-1f1d2c444098 None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:36.271015 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c79b9f2-a612-4dce-bac1-1f1d2c444098 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:36.271817 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c79b9f2-a612-4dce-bac1-1f1d2c444098 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:36.274154 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c79b9f2-a612-4dce-bac1-1f1d2c444098 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:36.286363 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b58678-8168-4b1b-b2c6-b59fa5a07616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:36.287489 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b58678-8168-4b1b-b2c6-b59fa5a07616 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:36.287896 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b58678-8168-4b1b-b2c6-b59fa5a07616 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:36.288371 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b58678-8168-4b1b-b2c6-b59fa5a07616 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:36.289433 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b58678-8168-4b1b-b2c6-b59fa5a07616 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:36.291452 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b58678-8168-4b1b-b2c6-b59fa5a07616 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:36.295478 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 419/850] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 24 06:22:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:36.297466 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c79b9f2-a612-4dce-bac1-1f1d2c444098 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:36.304002 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-a9b58678-8168-4b1b-b2c6-b59fa5a07616 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 06:22:36.304002 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:36.304002 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 06:22:36.304002 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 06:22:36.304002 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 06:22:36.304002 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 06:22:36.304002 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 06:22:36.304002 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:22:36.304002 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 06:22:36.304002 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:36.304002 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 06:22:36.304002 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:36.304002 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:36.304002 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:36.304002 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:36.304002 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:36.306044 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:36.306044 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:36.306044 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:36.306044 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:36.306044 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:36.306044 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:36.306044 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:36.306044 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 06:22:36.306044 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 06:22:36.306044 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 06:22:36.306044 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 06:22:36.306044 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:36.306044 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:36.306044 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:36.306044 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:36.306044 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:36.306044 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:36.307131 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:36.307131 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:36.307131 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:36.307131 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:36.307131 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:36.307131 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:36.307131 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:36.307131 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:36.307131 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:36.307131 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:36.307131 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 06:22:36.307131 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:22:36.307131 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 06:22:36.307131 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:36.310222 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 432/851] 10.222.0.47 () {56 vars in 1132 bytes} [Wed Aug 24 06:22:36 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:22:36.316464 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b58678-8168-4b1b-b2c6-b59fa5a07616 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:36.448193 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c79b9f2-a612-4dce-bac1-1f1d2c444098 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:36.451769 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:36.470614 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b58678-8168-4b1b-b2c6-b59fa5a07616 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:36.484826 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:36.487956 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4c17f0-b218-4ff6-ab69-3518a4aca201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:36.488605 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4c17f0-b218-4ff6-ab69-3518a4aca201 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:36.488878 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4c17f0-b218-4ff6-ab69-3518a4aca201 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:36.489205 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4c17f0-b218-4ff6-ab69-3518a4aca201 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:36.498023 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4c17f0-b218-4ff6-ab69-3518a4aca201 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:36.499603 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4c17f0-b218-4ff6-ab69-3518a4aca201 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:36.513558 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-9f4c17f0-b218-4ff6-ab69-3518a4aca201 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:22:36.513558 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:36.513558 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:36.513558 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:36.513558 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:36.513558 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:36.513558 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:36.513558 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:36.513558 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:36.513558 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:36.513558 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:36.513558 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:36.513558 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 06:22:36.513558 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 06:22:36.513558 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 06:22:36.514674 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 06:22:36.514674 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:36.514674 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:36.514674 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:36.514674 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:36.514674 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:36.514674 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:36.514674 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:36.514674 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:36.514674 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:36.514674 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:36.514674 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:36.514674 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:36.514674 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:36.514674 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:36.514674 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:36.514674 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:36.515563 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 06:22:36.515563 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 06:22:36.515563 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:22:36.515563 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:22:36.515563 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:22:36.515563 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:22:36.515563 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:22:36.515563 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:36.520687 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 420/852] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 24 06:22:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:36.522455 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab04a69c-aed1-4049-9633-060242eaab47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:36.523016 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4c17f0-b218-4ff6-ab69-3518a4aca201 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:36.523674 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab04a69c-aed1-4049-9633-060242eaab47 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:36.524454 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab04a69c-aed1-4049-9633-060242eaab47 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:36.525123 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab04a69c-aed1-4049-9633-060242eaab47 None admin] PATH_INFO: `/v3/services` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:36.526232 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab04a69c-aed1-4049-9633-060242eaab47 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:36.526754 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab04a69c-aed1-4049-9633-060242eaab47 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:36.529291 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab04a69c-aed1-4049-9633-060242eaab47 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:36.541543 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 433/853] 10.222.0.47 () {56 vars in 1126 bytes} [Wed Aug 24 06:22:36 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:36.542680 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab04a69c-aed1-4049-9633-060242eaab47 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:36.646585 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4c17f0-b218-4ff6-ab69-3518a4aca201 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:36.674627 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab04a69c-aed1-4049-9633-060242eaab47 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:36.686919 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f613625-1b0f-43e9-b460-dc30c36335fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:36.687541 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f613625-1b0f-43e9-b460-dc30c36335fb None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:36.687817 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f613625-1b0f-43e9-b460-dc30c36335fb None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:36.688084 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f613625-1b0f-43e9-b460-dc30c36335fb None admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:36.688972 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f613625-1b0f-43e9-b460-dc30c36335fb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:36.689209 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f613625-1b0f-43e9-b460-dc30c36335fb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:36.690709 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f613625-1b0f-43e9-b460-dc30c36335fb None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:36.709597 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 421/854] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Aug 24 06:22:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:36.711284 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f613625-1b0f-43e9-b460-dc30c36335fb None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:36.717398 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f680401-5c23-4f47-a76d-bb6c7511fcc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:36.718033 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f680401-5c23-4f47-a76d-bb6c7511fcc1 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:36.718325 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f680401-5c23-4f47-a76d-bb6c7511fcc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:36.718605 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f680401-5c23-4f47-a76d-bb6c7511fcc1 None admin] PATH_INFO: `/v3/services` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:36.719238 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f680401-5c23-4f47-a76d-bb6c7511fcc1 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:36.719541 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f680401-5c23-4f47-a76d-bb6c7511fcc1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:36.723865 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f680401-5c23-4f47-a76d-bb6c7511fcc1 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:36.726446 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 434/855] 10.222.0.47 () {56 vars in 1126 bytes} [Wed Aug 24 06:22:36 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:36.728454 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284ce4c2-611e-4d95-b489-45cc46719892 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:36.728743 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284ce4c2-611e-4d95-b489-45cc46719892 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:36.729019 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284ce4c2-611e-4d95-b489-45cc46719892 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:36.737366 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 435/856] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:22:36 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:22:36.743092 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282aabb6-691e-4e0a-bdcc-63f145a1a158 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:36.743972 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282aabb6-691e-4e0a-bdcc-63f145a1a158 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:36.744585 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282aabb6-691e-4e0a-bdcc-63f145a1a158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:36.747599 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:36.765269 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-282aabb6-691e-4e0a-bdcc-63f145a1a158 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:22:36.774569 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-282aabb6-691e-4e0a-bdcc-63f145a1a158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:22:36.807780 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:36.852227 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f613625-1b0f-43e9-b460-dc30c36335fb None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:36.861361 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 436/857] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:22:36 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:22:36.864117 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35a16b8-1be5-4738-a675-3ff2652c9dde None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:36.864526 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35a16b8-1be5-4738-a675-3ff2652c9dde None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:36.864909 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35a16b8-1be5-4738-a675-3ff2652c9dde None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:36.869406 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 437/858] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:22:36 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:22:36.875482 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2487307a-0d76-4064-a5bf-032f13f6f2e2 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:36.875807 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2487307a-0d76-4064-a5bf-032f13f6f2e2 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:36.876039 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2487307a-0d76-4064-a5bf-032f13f6f2e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:36.901065 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-2487307a-0d76-4064-a5bf-032f13f6f2e2 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:22:36.902350 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-2487307a-0d76-4064-a5bf-032f13f6f2e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:22:36.903217 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de529d62-ca3f-4c0e-ac0a-1cb01cd09f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:36.903943 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de529d62-ca3f-4c0e-ac0a-1cb01cd09f64 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:36.904224 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de529d62-ca3f-4c0e-ac0a-1cb01cd09f64 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:36.904727 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de529d62-ca3f-4c0e-ac0a-1cb01cd09f64 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:36.906483 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de529d62-ca3f-4c0e-ac0a-1cb01cd09f64 None admin] RBAC: Adding query filter params (role.id=486144ad1bcd417e86acdbc1d14327bf, scope.project.id=c02e49a89cdd40d885bfa816f06c58a1, user.id=02f1cbaf2cd6427585f298c9d58c319a) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:36.906820 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de529d62-ca3f-4c0e-ac0a-1cb01cd09f64 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:36.908280 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de529d62-ca3f-4c0e-ac0a-1cb01cd09f64 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:36.917228 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 422/859] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 24 06:22:36 2022] GET /identity/v3/role_assignments?user.id=02f1cbaf2cd6427585f298c9d58c319a&scope.project.id=c02e49a89cdd40d885bfa816f06c58a1&role.id=486144ad1bcd417e86acdbc1d14327bf => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:22:36.994070 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 438/860] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:22:36 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:22:37.142464 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de529d62-ca3f-4c0e-ac0a-1cb01cd09f64 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:37.244683 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:37.258438 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de529d62-ca3f-4c0e-ac0a-1cb01cd09f64 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:37.295966 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2487307a-0d76-4064-a5bf-032f13f6f2e2 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:37.302771 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34eeaaf-5149-41bf-9108-381adc12b2dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:37.303792 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34eeaaf-5149-41bf-9108-381adc12b2dc None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:37.304213 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34eeaaf-5149-41bf-9108-381adc12b2dc None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:37.304705 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34eeaaf-5149-41bf-9108-381adc12b2dc None admin] PATH_INFO: `/v3/roles/service` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:37.326218 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34eeaaf-5149-41bf-9108-381adc12b2dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:22:37.334351 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-e34eeaaf-5149-41bf-9108-381adc12b2dc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:22:37.334351 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:37.334351 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:37.334351 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:37.334351 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:37.334351 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:37.334351 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:37.334351 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:37.334351 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:37.334351 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:37.334351 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:37.334351 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:37.334351 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:22:37.334351 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:22:37.334351 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:22:37.334351 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:22:37.335343 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:22:37.335343 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:22:37.335343 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:22:37.335343 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:22:37.335343 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:37.335343 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:37.335343 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:37.335343 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:37.335343 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:37.335343 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:37.335343 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:37.335343 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:37.335343 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:37.335343 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:37.335343 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:37.335343 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:37.336264 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:37.336264 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:37.336264 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:37.336264 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:37.336264 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:22:37.336264 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:22:37.336264 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:22:37.336264 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:22:37.336264 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:22:37.336264 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:22:37.336264 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:22:37.336264 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:37.338816 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 423/861] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Aug 24 06:22:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:37.344854 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34eeaaf-5149-41bf-9108-381adc12b2dc None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:37.414299 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:37.421323 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2487307a-0d76-4064-a5bf-032f13f6f2e2 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:37.459932 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fceba57-4773-45d0-b484-4d15036b5011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:37.461517 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fceba57-4773-45d0-b484-4d15036b5011 None admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:37.462000 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fceba57-4773-45d0-b484-4d15036b5011 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:37.462480 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fceba57-4773-45d0-b484-4d15036b5011 None admin] PATH_INFO: `/v3/services` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:37.463190 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34eeaaf-5149-41bf-9108-381adc12b2dc None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:37.463574 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fceba57-4773-45d0-b484-4d15036b5011 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:37.465168 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fceba57-4773-45d0-b484-4d15036b5011 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:37.500369 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95df838e-1a1f-4ed1-afd9-0f6319190285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:37.501056 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95df838e-1a1f-4ed1-afd9-0f6319190285 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:37.501292 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95df838e-1a1f-4ed1-afd9-0f6319190285 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:37.501564 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95df838e-1a1f-4ed1-afd9-0f6319190285 None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:37.502317 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95df838e-1a1f-4ed1-afd9-0f6319190285 None admin] RBAC: Adding query filter params (name=service) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:37.502611 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95df838e-1a1f-4ed1-afd9-0f6319190285 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:37.504024 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95df838e-1a1f-4ed1-afd9-0f6319190285 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:37.514936 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 424/862] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Aug 24 06:22:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:22:37.520659 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95df838e-1a1f-4ed1-afd9-0f6319190285 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:37.540200 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:37.540606 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 439/863] 10.222.0.47 () {60 vars in 1141 bytes} [Wed Aug 24 06:22:37 2022] POST /identity/v3/services => generated 251 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:22:37.649938 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95df838e-1a1f-4ed1-afd9-0f6319190285 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:37.696904 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b6ab75-3d90-4475-a3bc-219d51bf21e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:37.697466 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b6ab75-3d90-4475-a3bc-219d51bf21e0 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:37.697740 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b6ab75-3d90-4475-a3bc-219d51bf21e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:37.698009 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b6ab75-3d90-4475-a3bc-219d51bf21e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:37.711019 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b6ab75-3d90-4475-a3bc-219d51bf21e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:37.712670 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b6ab75-3d90-4475-a3bc-219d51bf21e0 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:37.730861 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-86b6ab75-3d90-4475-a3bc-219d51bf21e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:37.730861 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:37.730861 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:37.730861 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:37.730861 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:37.730861 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:37.730861 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:37.730861 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:37.730861 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:37.730861 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:37.730861 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:37.730861 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:37.730861 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:22:37.730861 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:22:37.730861 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:22:37.730861 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:22:37.731867 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:37.731867 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:37.731867 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:37.731867 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:37.731867 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:37.731867 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:37.731867 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:37.731867 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:37.731867 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:37.731867 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:37.731867 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:37.731867 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:37.731867 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:37.731867 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:37.731867 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:37.731867 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:37.731867 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:22:37.732861 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:22:37.732861 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:37.732861 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:37.732861 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 425/864] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:22:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:37.739605 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fceba57-4773-45d0-b484-4d15036b5011 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:37.865207 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fceba57-4773-45d0-b484-4d15036b5011 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:37.911226 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a0937f-1bb1-431f-8bba-75b6e7e98cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:37.911843 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a0937f-1bb1-431f-8bba-75b6e7e98cb6 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:37.912120 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a0937f-1bb1-431f-8bba-75b6e7e98cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:37.912388 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a0937f-1bb1-431f-8bba-75b6e7e98cb6 None admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:37.913033 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a0937f-1bb1-431f-8bba-75b6e7e98cb6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:37.913363 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a0937f-1bb1-431f-8bba-75b6e7e98cb6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:37.915088 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a0937f-1bb1-431f-8bba-75b6e7e98cb6 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:37.928753 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 440/865] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 24 06:22:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:37.937397 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b6ab75-3d90-4475-a3bc-219d51bf21e0 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:38.011714 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:38.063445 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b6ab75-3d90-4475-a3bc-219d51bf21e0 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:38.100129 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d028a7a-63ae-4e58-bed5-d5e84313c51d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:38.101151 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d028a7a-63ae-4e58-bed5-d5e84313c51d None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:38.102214 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d028a7a-63ae-4e58-bed5-d5e84313c51d None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:38.102768 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d028a7a-63ae-4e58-bed5-d5e84313c51d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:38.111352 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d028a7a-63ae-4e58-bed5-d5e84313c51d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:38.113222 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d028a7a-63ae-4e58-bed5-d5e84313c51d None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:38.126620 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-2d028a7a-63ae-4e58-bed5-d5e84313c51d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 06:22:38.126620 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:38.126620 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:38.126620 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:38.126620 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:38.126620 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:38.126620 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:38.126620 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:38.126620 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:38.126620 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:38.126620 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:38.126620 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:38.126620 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 06:22:38.126620 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 06:22:38.126620 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 06:22:38.126620 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 06:22:38.127663 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:38.127663 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:38.127663 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 06:22:38.127663 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:22:38.127663 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 06:22:38.127663 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:22:38.127663 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:38.127663 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:38.127663 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:38.127663 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:38.127663 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:38.127663 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:38.127663 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:38.127663 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:38.127663 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:38.127663 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:38.127663 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:38.127663 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:38.128600 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:38.128600 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:38.128600 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 06:22:38.128600 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 06:22:38.128600 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 06:22:38.128600 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 06:22:38.128600 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 06:22:38.128600 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 06:22:38.128600 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 06:22:38.128600 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:38.131447 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 426/866] 10.222.0.47 () {56 vars in 1117 bytes} [Wed Aug 24 06:22:37 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:38.137908 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a0937f-1bb1-431f-8bba-75b6e7e98cb6 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:38.158531 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:38.249278 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a0937f-1bb1-431f-8bba-75b6e7e98cb6 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:38.290092 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88b78be-13af-45ef-8f03-665e071eee32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:38.290674 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88b78be-13af-45ef-8f03-665e071eee32 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:38.290956 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88b78be-13af-45ef-8f03-665e071eee32 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:38.291221 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88b78be-13af-45ef-8f03-665e071eee32 None admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:38.292184 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88b78be-13af-45ef-8f03-665e071eee32 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:38.292628 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88b78be-13af-45ef-8f03-665e071eee32 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:38.294503 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88b78be-13af-45ef-8f03-665e071eee32 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:38.314526 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 441/867] 10.222.0.47 () {56 vars in 1149 bytes} [Wed Aug 24 06:22:38 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:38.319916 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d028a7a-63ae-4e58-bed5-d5e84313c51d None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:38.447074 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d028a7a-63ae-4e58-bed5-d5e84313c51d None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:38.472000 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:38.494516 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a0b306-7923-4a96-b52c-d81eb266d5a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:38.495151 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a0b306-7923-4a96-b52c-d81eb266d5a9 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:38.495434 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a0b306-7923-4a96-b52c-d81eb266d5a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:38.495712 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a0b306-7923-4a96-b52c-d81eb266d5a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:38.571984 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a0b306-7923-4a96-b52c-d81eb266d5a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:38.573402 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a0b306-7923-4a96-b52c-d81eb266d5a9 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:38.682936 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 427/868] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 24 06:22:38 2022] GET /identity/v3/auth/tokens => generated 1262 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 06:22:38.690376 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88b78be-13af-45ef-8f03-665e071eee32 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:38.779568 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:38.809807 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88b78be-13af-45ef-8f03-665e071eee32 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:38.851863 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad01ce3-252f-4d78-b9f2-5fdb1b3935be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:38.853026 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad01ce3-252f-4d78-b9f2-5fdb1b3935be None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:38.853548 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad01ce3-252f-4d78-b9f2-5fdb1b3935be None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:38.854041 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad01ce3-252f-4d78-b9f2-5fdb1b3935be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:38.865159 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad01ce3-252f-4d78-b9f2-5fdb1b3935be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:38.867014 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad01ce3-252f-4d78-b9f2-5fdb1b3935be None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:38.883068 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2183e3f8-dd1a-40a1-9e73-329f41bb7c44 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:38.883332 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2183e3f8-dd1a-40a1-9e73-329f41bb7c44 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:38.883769 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2183e3f8-dd1a-40a1-9e73-329f41bb7c44 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:38.885719 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-0ad01ce3-252f-4d78-b9f2-5fdb1b3935be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:38.885719 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:38.885719 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:38.885719 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:38.885719 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:38.885719 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:38.885719 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:38.885719 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:38.885719 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:38.885719 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:38.885719 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:38.885719 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:38.885719 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:22:38.885719 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:22:38.885719 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:22:38.885719 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:22:38.886628 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:38.886628 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:38.886628 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:38.886628 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:38.886628 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:38.886628 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:38.886628 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:38.886628 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:38.886628 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:38.886628 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:38.886628 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:38.886628 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:38.886628 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:38.886628 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:38.886628 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:38.886628 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:38.886628 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:22:38.887514 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:22:38.887514 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:38.887514 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:38.887514 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 442/869] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:22:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:38.889485 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 428/870] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:22:38 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 06:22:38.890778 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adc5f43-8018-4fca-961a-8eecdd1418ab None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:38.891066 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adc5f43-8018-4fca-961a-8eecdd1418ab None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:38.891338 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adc5f43-8018-4fca-961a-8eecdd1418ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:38.895267 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2183e3f8-dd1a-40a1-9e73-329f41bb7c44 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:38.910591 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-2adc5f43-8018-4fca-961a-8eecdd1418ab None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:22:38.912127 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-2adc5f43-8018-4fca-961a-8eecdd1418ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:22:38.921445 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:39.013236 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 443/871] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:22:38 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:22:39.033527 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2183e3f8-dd1a-40a1-9e73-329f41bb7c44 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:39.068074 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b1d041-ef5c-4316-9205-fe456abf575a None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:39.068396 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b1d041-ef5c-4316-9205-fe456abf575a None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:39.068615 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b1d041-ef5c-4316-9205-fe456abf575a None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:39.071907 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 444/872] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:22:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 06:22:39.072872 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf76fff6-8f1c-4f80-8d6c-8b6b73f3c84b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:39.073696 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf76fff6-8f1c-4f80-8d6c-8b6b73f3c84b None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:39.073976 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf76fff6-8f1c-4f80-8d6c-8b6b73f3c84b None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:39.074418 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf76fff6-8f1c-4f80-8d6c-8b6b73f3c84b None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:39.075184 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf76fff6-8f1c-4f80-8d6c-8b6b73f3c84b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:39.075593 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf76fff6-8f1c-4f80-8d6c-8b6b73f3c84b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:39.077707 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf76fff6-8f1c-4f80-8d6c-8b6b73f3c84b None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:39.078233 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5fb094-4334-40a9-9bc5-d02f8e26efce None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:39.078571 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5fb094-4334-40a9-9bc5-d02f8e26efce None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:39.078844 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5fb094-4334-40a9-9bc5-d02f8e26efce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:39.094082 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 429/873] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 24 06:22:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:39.099593 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf76fff6-8f1c-4f80-8d6c-8b6b73f3c84b None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:39.103029 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-da5fb094-4334-40a9-9bc5-d02f8e26efce None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:22:39.104186 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-da5fb094-4334-40a9-9bc5-d02f8e26efce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:22:39.178904 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:39.196153 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:39.204817 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 445/874] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:22:39 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 06:22:39.229413 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf76fff6-8f1c-4f80-8d6c-8b6b73f3c84b None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:39.269371 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eda6f5b-2435-4eef-8cba-4ac30fb38818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:39.270096 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eda6f5b-2435-4eef-8cba-4ac30fb38818 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:39.270433 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eda6f5b-2435-4eef-8cba-4ac30fb38818 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:39.270716 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eda6f5b-2435-4eef-8cba-4ac30fb38818 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:39.278991 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eda6f5b-2435-4eef-8cba-4ac30fb38818 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:39.280600 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eda6f5b-2435-4eef-8cba-4ac30fb38818 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:39.292349 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5fb094-4334-40a9-9bc5-d02f8e26efce None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:39.293262 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-5eda6f5b-2435-4eef-8cba-4ac30fb38818 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:22:39.293262 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:39.293262 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:39.293262 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:39.293262 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:39.293262 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:39.293262 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:39.293262 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:39.293262 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:39.293262 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:39.293262 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:39.293262 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:39.293262 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 06:22:39.293262 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 06:22:39.293262 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 06:22:39.294135 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 06:22:39.294135 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:39.294135 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:39.294135 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:39.294135 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:39.294135 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:39.294135 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:39.294135 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:39.294135 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:39.294135 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:39.294135 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:39.294135 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:39.294135 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:39.294135 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:39.294135 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:39.294135 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:39.294135 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:39.295030 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 06:22:39.295030 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 06:22:39.295030 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:22:39.295030 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:22:39.295030 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:22:39.295030 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:22:39.295030 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:22:39.295030 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:39.295030 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 430/875] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 24 06:22:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:22:39.300765 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eda6f5b-2435-4eef-8cba-4ac30fb38818 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:39.426943 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eda6f5b-2435-4eef-8cba-4ac30fb38818 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:39.427456 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5fb094-4334-40a9-9bc5-d02f8e26efce None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:39.465247 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c48aaa3-c736-4ee2-bbbb-a02fd6ae3a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:39.466079 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c48aaa3-c736-4ee2-bbbb-a02fd6ae3a65 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:39.466272 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c48aaa3-c736-4ee2-bbbb-a02fd6ae3a65 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:39.466532 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c48aaa3-c736-4ee2-bbbb-a02fd6ae3a65 None admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:39.467228 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c48aaa3-c736-4ee2-bbbb-a02fd6ae3a65 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:39.467599 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c48aaa3-c736-4ee2-bbbb-a02fd6ae3a65 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:39.468952 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c48aaa3-c736-4ee2-bbbb-a02fd6ae3a65 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:39.475882 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f848ae55-16a8-4b5f-a57a-f133e4c2990c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:39.477348 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:39.478166 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f848ae55-16a8-4b5f-a57a-f133e4c2990c None admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:39.478166 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f848ae55-16a8-4b5f-a57a-f133e4c2990c None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:39.478454 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f848ae55-16a8-4b5f-a57a-f133e4c2990c None admin] PATH_INFO: `/v3/services` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:39.479054 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f848ae55-16a8-4b5f-a57a-f133e4c2990c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:39.480435 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f848ae55-16a8-4b5f-a57a-f133e4c2990c None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:39.487001 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 431/876] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Aug 24 06:22:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:22:39.495085 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c48aaa3-c736-4ee2-bbbb-a02fd6ae3a65 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:39.573400 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 446/877] 10.222.0.47 () {60 vars in 1141 bytes} [Wed Aug 24 06:22:39 2022] POST /identity/v3/services => generated 269 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:22:39.574598 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f848ae55-16a8-4b5f-a57a-f133e4c2990c None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:39.611506 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c48aaa3-c736-4ee2-bbbb-a02fd6ae3a65 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:39.641658 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:39.660932 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d917a0c4-faa5-47c2-a43c-98f569f2d6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:39.662043 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d917a0c4-faa5-47c2-a43c-98f569f2d6c7 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:39.662519 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d917a0c4-faa5-47c2-a43c-98f569f2d6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:39.662945 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d917a0c4-faa5-47c2-a43c-98f569f2d6c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:39.664554 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d917a0c4-faa5-47c2-a43c-98f569f2d6c7 None admin] RBAC: Adding query filter params (role.id=1712d85aedaf4189bf4bc89094d888c3, scope.project.id=c02e49a89cdd40d885bfa816f06c58a1, user.id=ea70acd744994250a2905b5dd7163dfb) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:39.665251 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d917a0c4-faa5-47c2-a43c-98f569f2d6c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:39.667104 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d917a0c4-faa5-47c2-a43c-98f569f2d6c7 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:39.674571 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 432/878] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 24 06:22:39 2022] GET /identity/v3/role_assignments?user.id=ea70acd744994250a2905b5dd7163dfb&scope.project.id=c02e49a89cdd40d885bfa816f06c58a1&role.id=1712d85aedaf4189bf4bc89094d888c3 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:22:39.703343 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2c6947-a07d-49d0-ba6c-e6796d68d221 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:39.703529 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2c6947-a07d-49d0-ba6c-e6796d68d221 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:39.703808 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2c6947-a07d-49d0-ba6c-e6796d68d221 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:39.709644 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 433/879] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:22:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 06:22:39.714820 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6faf17f-3f63-4a9a-b074-1148049e1281 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:39.714820 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6faf17f-3f63-4a9a-b074-1148049e1281 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:39.714820 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6faf17f-3f63-4a9a-b074-1148049e1281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:39.729340 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f848ae55-16a8-4b5f-a57a-f133e4c2990c None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:39.735984 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-a6faf17f-3f63-4a9a-b074-1148049e1281 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:22:39.737638 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-a6faf17f-3f63-4a9a-b074-1148049e1281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:22:39.779082 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5055ba-8bad-483f-8130-e86b594db9fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:39.780175 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5055ba-8bad-483f-8130-e86b594db9fa None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:39.781495 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5055ba-8bad-483f-8130-e86b594db9fa None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:39.781947 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5055ba-8bad-483f-8130-e86b594db9fa None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:39.782961 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5055ba-8bad-483f-8130-e86b594db9fa None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:39.785137 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5055ba-8bad-483f-8130-e86b594db9fa None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:39.795484 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-8f5055ba-8bad-483f-8130-e86b594db9fa None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 06:22:39.795484 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:39.795484 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 06:22:39.795484 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 06:22:39.795484 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 06:22:39.795484 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 06:22:39.795484 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 06:22:39.795484 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:22:39.795484 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 06:22:39.795484 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:39.795484 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 06:22:39.795484 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:39.795484 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:39.795484 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:39.795484 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:39.795484 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:39.795484 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:39.796563 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:39.796563 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:39.796563 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:39.796563 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:39.796563 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:39.796563 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:39.796563 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 06:22:39.796563 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 06:22:39.796563 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 06:22:39.796563 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 06:22:39.796563 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:39.796563 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:39.796563 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:39.796563 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:39.796563 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:39.796563 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:39.797500 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:39.797500 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:39.797500 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:39.797500 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:39.797500 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:39.797500 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:39.797500 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:39.797500 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:39.797500 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:39.797500 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:39.797500 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 06:22:39.797500 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:22:39.797500 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 06:22:39.797500 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:39.800745 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 447/880] 10.222.0.47 () {56 vars in 1126 bytes} [Wed Aug 24 06:22:39 2022] GET /identity/v3/services/object-store => generated 93 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:39.803131 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5055ba-8bad-483f-8130-e86b594db9fa None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:39.826120 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:39.842794 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 434/881] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:22:39 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:22:39.927114 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5055ba-8bad-483f-8130-e86b594db9fa None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:39.939320 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:39.967173 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f72d01-46b4-4219-a547-36275a3a910d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:39.967719 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f72d01-46b4-4219-a547-36275a3a910d None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:39.967998 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f72d01-46b4-4219-a547-36275a3a910d None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:39.968268 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f72d01-46b4-4219-a547-36275a3a910d None admin] PATH_INFO: `/v3/services` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:39.968898 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f72d01-46b4-4219-a547-36275a3a910d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:39.969238 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f72d01-46b4-4219-a547-36275a3a910d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:39.970582 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f72d01-46b4-4219-a547-36275a3a910d None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:39.976021 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 448/882] 10.222.0.47 () {56 vars in 1122 bytes} [Wed Aug 24 06:22:39 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:22:39.982232 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6faf17f-3f63-4a9a-b074-1148049e1281 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:40.101343 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6faf17f-3f63-4a9a-b074-1148049e1281 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:40.115305 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f72d01-46b4-4219-a547-36275a3a910d None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:40.140108 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38752913-e509-4b3d-953b-7e92c19f2147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:40.140825 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38752913-e509-4b3d-953b-7e92c19f2147 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:40.141106 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38752913-e509-4b3d-953b-7e92c19f2147 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:40.141404 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38752913-e509-4b3d-953b-7e92c19f2147 None admin] PATH_INFO: `/v3/services` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:40.142041 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38752913-e509-4b3d-953b-7e92c19f2147 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:40.142340 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38752913-e509-4b3d-953b-7e92c19f2147 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:40.143863 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38752913-e509-4b3d-953b-7e92c19f2147 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:40.157347 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 435/883] 10.222.0.47 () {56 vars in 1122 bytes} [Wed Aug 24 06:22:39 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:40.234679 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f72d01-46b4-4219-a547-36275a3a910d None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:40.274127 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6516c56-c9b4-4226-872e-51fe1c593804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:40.274795 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6516c56-c9b4-4226-872e-51fe1c593804 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:40.275069 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6516c56-c9b4-4226-872e-51fe1c593804 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:40.275337 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6516c56-c9b4-4226-872e-51fe1c593804 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:40.275947 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6516c56-c9b4-4226-872e-51fe1c593804 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:40.277410 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6516c56-c9b4-4226-872e-51fe1c593804 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:40.287210 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-e6516c56-c9b4-4226-872e-51fe1c593804 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 06:22:40.287210 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:40.287210 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 06:22:40.287210 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 06:22:40.287210 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 06:22:40.287210 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 06:22:40.287210 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 06:22:40.287210 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:22:40.287210 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 06:22:40.287210 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:40.287210 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 06:22:40.287210 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:40.287210 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:40.287210 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:40.287210 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:40.287210 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:40.287210 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:40.288324 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:40.288324 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:40.288324 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:40.288324 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:40.288324 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:40.288324 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:40.288324 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 06:22:40.288324 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 06:22:40.288324 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 06:22:40.288324 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 06:22:40.288324 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:40.288324 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:40.288324 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:40.288324 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:40.288324 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:40.288324 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:40.289354 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:40.289354 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:40.289354 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:40.289354 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:40.289354 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:40.289354 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:40.289354 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:40.289354 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:40.289354 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:40.289354 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:40.289354 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 06:22:40.289354 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:22:40.289354 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 06:22:40.289354 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:40.289354 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 449/884] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Aug 24 06:22:40 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:22:40.294963 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38752913-e509-4b3d-953b-7e92c19f2147 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:40.359338 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:40.409451 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38752913-e509-4b3d-953b-7e92c19f2147 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:40.450333 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd325529-ab5e-4f7f-bde4-f6faffa8ffd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:40.451355 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd325529-ab5e-4f7f-bde4-f6faffa8ffd4 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:40.451837 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd325529-ab5e-4f7f-bde4-f6faffa8ffd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:40.452437 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd325529-ab5e-4f7f-bde4-f6faffa8ffd4 None admin] PATH_INFO: `/v3/services` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:40.453895 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd325529-ab5e-4f7f-bde4-f6faffa8ffd4 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:40.454734 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd325529-ab5e-4f7f-bde4-f6faffa8ffd4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:40.457415 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd325529-ab5e-4f7f-bde4-f6faffa8ffd4 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:40.464270 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 436/885] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Aug 24 06:22:40 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:40.469748 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6516c56-c9b4-4226-872e-51fe1c593804 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:40.514244 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:40.582116 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6516c56-c9b4-4226-872e-51fe1c593804 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:40.618893 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c2eede-962e-41e9-92f7-592083557676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:40.619522 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c2eede-962e-41e9-92f7-592083557676 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:40.619967 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c2eede-962e-41e9-92f7-592083557676 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:40.620255 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c2eede-962e-41e9-92f7-592083557676 None admin] PATH_INFO: `/v3/services` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:40.620904 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c2eede-962e-41e9-92f7-592083557676 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:40.621229 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c2eede-962e-41e9-92f7-592083557676 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:40.622661 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c2eede-962e-41e9-92f7-592083557676 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:40.628447 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 450/886] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Aug 24 06:22:40 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:40.634543 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd325529-ab5e-4f7f-bde4-f6faffa8ffd4 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:40.753750 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd325529-ab5e-4f7f-bde4-f6faffa8ffd4 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:40.773236 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:40.792330 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a8ead5-73e6-4a42-b75a-30cb9e337b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:40.792958 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a8ead5-73e6-4a42-b75a-30cb9e337b7b None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:40.793288 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a8ead5-73e6-4a42-b75a-30cb9e337b7b None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:40.793571 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a8ead5-73e6-4a42-b75a-30cb9e337b7b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:40.794288 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a8ead5-73e6-4a42-b75a-30cb9e337b7b None admin] RBAC: Adding query filter params (interface=public, service_id=2e175603f98b4bb8b2aca03f17456bfe, region_id=RegionOne) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:40.794614 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a8ead5-73e6-4a42-b75a-30cb9e337b7b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:40.795960 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a8ead5-73e6-4a42-b75a-30cb9e337b7b None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:40.802014 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 437/887] 10.222.0.47 () {56 vars in 1251 bytes} [Wed Aug 24 06:22:40 2022] GET /identity/v3/endpoints?service_id=2e175603f98b4bb8b2aca03f17456bfe&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:40.808423 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c2eede-962e-41e9-92f7-592083557676 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:40.922661 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c2eede-962e-41e9-92f7-592083557676 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:40.924262 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:40.961329 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4133b662-dcae-41eb-a228-e8835c8a4cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:40.961976 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4133b662-dcae-41eb-a228-e8835c8a4cff None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:40.962263 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4133b662-dcae-41eb-a228-e8835c8a4cff None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:40.962546 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4133b662-dcae-41eb-a228-e8835c8a4cff None admin] PATH_INFO: `/v3/services` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:40.963179 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4133b662-dcae-41eb-a228-e8835c8a4cff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:40.965255 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4133b662-dcae-41eb-a228-e8835c8a4cff None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:40.972032 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 451/888] 10.222.0.47 () {56 vars in 1087 bytes} [Wed Aug 24 06:22:40 2022] GET /identity/v3/services => generated 1268 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:22:41.719779 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1849fa64-5c8c-4042-84d2-dbda7173df95 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:41.720748 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1849fa64-5c8c-4042-84d2-dbda7173df95 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:41.721266 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1849fa64-5c8c-4042-84d2-dbda7173df95 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:41.723095 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 438/889] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:22:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:22:41.729555 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040386cd-b425-48a7-86f1-75c8e07bc6bd None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:41.730204 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040386cd-b425-48a7-86f1-75c8e07bc6bd None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:41.730680 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040386cd-b425-48a7-86f1-75c8e07bc6bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:41.750555 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-040386cd-b425-48a7-86f1-75c8e07bc6bd None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:22:41.752122 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-040386cd-b425-48a7-86f1-75c8e07bc6bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:22:41.833842 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 452/890] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:22:41 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:22:41.903454 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88862da1-341c-42a1-ac1c-e9b2b447a728 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:41.903744 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88862da1-341c-42a1-ac1c-e9b2b447a728 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:41.903933 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88862da1-341c-42a1-ac1c-e9b2b447a728 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:41.905494 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 439/891] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:22:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:22:41.910880 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fa5d20-97d4-43d9-9ccf-13d726dbe1c6 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:41.911166 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fa5d20-97d4-43d9-9ccf-13d726dbe1c6 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:41.911512 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fa5d20-97d4-43d9-9ccf-13d726dbe1c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:41.935328 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-78fa5d20-97d4-43d9-9ccf-13d726dbe1c6 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:22:41.936541 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-78fa5d20-97d4-43d9-9ccf-13d726dbe1c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:22:42.012265 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 453/892] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:22:41 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:22:42.133580 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88862da1-341c-42a1-ac1c-e9b2b447a728 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:42.261662 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88862da1-341c-42a1-ac1c-e9b2b447a728 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:42.270002 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42b24d0-a48f-422f-a152-291ea29f382e None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:42.270294 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42b24d0-a48f-422f-a152-291ea29f382e None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:42.270572 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42b24d0-a48f-422f-a152-291ea29f382e None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:42.272588 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 454/893] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:22:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:22:42.278871 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd89098-5f01-4474-924d-23143522f895 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:42.279152 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd89098-5f01-4474-924d-23143522f895 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:42.279429 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd89098-5f01-4474-924d-23143522f895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:42.299584 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-6dd89098-5f01-4474-924d-23143522f895 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:22:42.301118 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-6dd89098-5f01-4474-924d-23143522f895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:22:42.302877 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e1cf97-6c26-4f3b-b59b-b9ef3f45a3a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:42.303744 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e1cf97-6c26-4f3b-b59b-b9ef3f45a3a2 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:42.304279 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e1cf97-6c26-4f3b-b59b-b9ef3f45a3a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:42.304952 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e1cf97-6c26-4f3b-b59b-b9ef3f45a3a2 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:42.305989 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e1cf97-6c26-4f3b-b59b-b9ef3f45a3a2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:42.307582 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e1cf97-6c26-4f3b-b59b-b9ef3f45a3a2 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:42.317484 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-15e1cf97-6c26-4f3b-b59b-b9ef3f45a3a2 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 06:22:42.317484 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:42.317484 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 06:22:42.317484 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 06:22:42.317484 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 06:22:42.317484 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 06:22:42.317484 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 06:22:42.317484 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:22:42.317484 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 06:22:42.317484 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:42.317484 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 06:22:42.317484 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:42.317484 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:42.317484 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:42.317484 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:42.317484 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:42.318673 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:42.318673 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:42.318673 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:42.318673 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:42.318673 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:42.318673 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:42.318673 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:42.318673 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 06:22:42.318673 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 06:22:42.318673 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 06:22:42.318673 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 06:22:42.318673 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:42.318673 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:42.318673 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:42.318673 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:42.318673 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:42.318673 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:42.319863 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:42.319863 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:42.319863 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:42.319863 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:42.319863 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:42.319863 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:42.319863 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:42.319863 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:42.319863 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:42.319863 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:42.319863 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 06:22:42.319863 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:22:42.319863 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 06:22:42.319863 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:42.322769 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 440/894] 10.222.0.47 () {56 vars in 1132 bytes} [Wed Aug 24 06:22:42 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:42.324720 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:42.331874 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e1cf97-6c26-4f3b-b59b-b9ef3f45a3a2 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:42.389260 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 455/895] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:22:42 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 06:22:42.389588 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd89098-5f01-4474-924d-23143522f895 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:42.401812 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:42.467160 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e1cf97-6c26-4f3b-b59b-b9ef3f45a3a2 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:42.505772 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a368cc4f-de9b-43c6-97c0-bf11d50bc665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:42.506766 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a368cc4f-de9b-43c6-97c0-bf11d50bc665 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:42.508146 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a368cc4f-de9b-43c6-97c0-bf11d50bc665 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:42.508450 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a368cc4f-de9b-43c6-97c0-bf11d50bc665 None admin] PATH_INFO: `/v3/services` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:42.509108 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a368cc4f-de9b-43c6-97c0-bf11d50bc665 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:42.509441 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a368cc4f-de9b-43c6-97c0-bf11d50bc665 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:42.510812 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a368cc4f-de9b-43c6-97c0-bf11d50bc665 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:42.515646 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd89098-5f01-4474-924d-23143522f895 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:42.517860 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 441/896] 10.222.0.47 () {56 vars in 1126 bytes} [Wed Aug 24 06:22:42 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:42.524430 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a368cc4f-de9b-43c6-97c0-bf11d50bc665 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:42.567702 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5079fa4e-e574-467e-aa03-56ac91dac4c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:42.568726 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5079fa4e-e574-467e-aa03-56ac91dac4c0 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:42.569511 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5079fa4e-e574-467e-aa03-56ac91dac4c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:42.571304 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5079fa4e-e574-467e-aa03-56ac91dac4c0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:42.585225 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5079fa4e-e574-467e-aa03-56ac91dac4c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:22:42.592116 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-5079fa4e-e574-467e-aa03-56ac91dac4c0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 06:22:42.592116 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:42.592116 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:42.592116 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:42.592116 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:42.592116 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:42.592116 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:42.592116 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:42.592116 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:42.592116 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:42.592116 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:42.592116 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:42.592116 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:22:42.592116 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:22:42.592116 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:22:42.592116 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:22:42.593038 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:22:42.593038 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:22:42.593038 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:22:42.593038 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:22:42.593038 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:42.593038 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:42.593038 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:42.593038 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:42.593038 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:42.593038 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:42.593038 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:42.593038 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:42.593038 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:42.593038 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:42.593038 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:42.593038 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:42.593953 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:42.593953 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:42.593953 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:42.593953 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:42.593953 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:22:42.593953 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:22:42.593953 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:22:42.593953 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:22:42.593953 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:22:42.593953 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:22:42.593953 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 06:22:42.593953 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:42.599399 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 456/897] 10.222.0.47 () {56 vars in 1120 bytes} [Wed Aug 24 06:22:42 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:42.602624 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5079fa4e-e574-467e-aa03-56ac91dac4c0 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:42.635683 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:42.664638 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a368cc4f-de9b-43c6-97c0-bf11d50bc665 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:42.674014 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:42.699675 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e28a1fb-f572-4eb7-a4b5-2d1a6fbd69cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:42.700293 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e28a1fb-f572-4eb7-a4b5-2d1a6fbd69cd None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:42.700575 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e28a1fb-f572-4eb7-a4b5-2d1a6fbd69cd None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:42.700850 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e28a1fb-f572-4eb7-a4b5-2d1a6fbd69cd None admin] PATH_INFO: `/v3/services` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:42.701524 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e28a1fb-f572-4eb7-a4b5-2d1a6fbd69cd None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:42.701828 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e28a1fb-f572-4eb7-a4b5-2d1a6fbd69cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:42.703183 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e28a1fb-f572-4eb7-a4b5-2d1a6fbd69cd None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:42.708242 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 442/898] 10.222.0.47 () {56 vars in 1126 bytes} [Wed Aug 24 06:22:42 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:22:42.709306 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e28a1fb-f572-4eb7-a4b5-2d1a6fbd69cd None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:42.738874 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5079fa4e-e574-467e-aa03-56ac91dac4c0 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:42.779099 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d367e5-454c-4c7b-9cdf-cea3adb24d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:42.779626 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d367e5-454c-4c7b-9cdf-cea3adb24d88 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:42.779918 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d367e5-454c-4c7b-9cdf-cea3adb24d88 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:42.780169 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d367e5-454c-4c7b-9cdf-cea3adb24d88 None admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:42.780827 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d367e5-454c-4c7b-9cdf-cea3adb24d88 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:42.781329 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d367e5-454c-4c7b-9cdf-cea3adb24d88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:42.782875 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d367e5-454c-4c7b-9cdf-cea3adb24d88 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:42.796850 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 457/899] 10.222.0.47 () {56 vars in 1115 bytes} [Wed Aug 24 06:22:42 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:42.796850 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d367e5-454c-4c7b-9cdf-cea3adb24d88 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:42.830965 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e28a1fb-f572-4eb7-a4b5-2d1a6fbd69cd None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:42.865611 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45368446-db52-407f-8da2-a8143d6d1260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:42.866224 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45368446-db52-407f-8da2-a8143d6d1260 None admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:42.866505 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45368446-db52-407f-8da2-a8143d6d1260 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:42.867406 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45368446-db52-407f-8da2-a8143d6d1260 None admin] PATH_INFO: `/v3/services` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:42.868046 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45368446-db52-407f-8da2-a8143d6d1260 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:42.869427 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45368446-db52-407f-8da2-a8143d6d1260 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:42.881399 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:42.903846 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d367e5-454c-4c7b-9cdf-cea3adb24d88 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:42.941453 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 443/900] 10.222.0.47 () {60 vars in 1141 bytes} [Wed Aug 24 06:22:42 2022] POST /identity/v3/services => generated 241 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:22:42.942543 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45368446-db52-407f-8da2-a8143d6d1260 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:42.956117 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d945b146-392a-450e-9f3b-25d78cd266f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:42.956117 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d945b146-392a-450e-9f3b-25d78cd266f7 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:42.956117 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d945b146-392a-450e-9f3b-25d78cd266f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:42.956117 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d945b146-392a-450e-9f3b-25d78cd266f7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:42.956621 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d945b146-392a-450e-9f3b-25d78cd266f7 None admin] RBAC: Adding query filter params (interface=public, service_id=6aa65db0f5fa497d89456541fc511449, region_id=RegionOne) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:42.956621 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d945b146-392a-450e-9f3b-25d78cd266f7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:42.956621 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d945b146-392a-450e-9f3b-25d78cd266f7 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:42.968840 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 458/901] 10.222.0.47 () {56 vars in 1251 bytes} [Wed Aug 24 06:22:42 2022] GET /identity/v3/endpoints?service_id=6aa65db0f5fa497d89456541fc511449&interface=public®ion_id=RegionOne => generated 194 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:42.974524 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d945b146-392a-450e-9f3b-25d78cd266f7 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:42.988817 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:43.073642 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45368446-db52-407f-8da2-a8143d6d1260 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:43.100729 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d945b146-392a-450e-9f3b-25d78cd266f7 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:43.115382 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd619869-8e8c-4289-85b2-cd7cdf645409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:43.116025 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd619869-8e8c-4289-85b2-cd7cdf645409 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:43.116309 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd619869-8e8c-4289-85b2-cd7cdf645409 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:43.117101 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd619869-8e8c-4289-85b2-cd7cdf645409 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:43.128089 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd619869-8e8c-4289-85b2-cd7cdf645409 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:43.129612 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd619869-8e8c-4289-85b2-cd7cdf645409 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:43.141875 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86583b0-9355-47c6-b522-b08b100af777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:43.142484 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86583b0-9355-47c6-b522-b08b100af777 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:43.142762 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86583b0-9355-47c6-b522-b08b100af777 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:43.143030 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86583b0-9355-47c6-b522-b08b100af777 None admin] PATH_INFO: `/v3/services` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:43.143626 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86583b0-9355-47c6-b522-b08b100af777 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:43.144976 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86583b0-9355-47c6-b522-b08b100af777 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:43.145706 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-cd619869-8e8c-4289-85b2-cd7cdf645409 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:43.145706 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:43.145706 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:43.145706 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:43.145706 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:43.145706 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:43.145706 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:43.145706 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:43.145706 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:43.145706 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:43.145706 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:43.145706 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:43.145706 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:22:43.145706 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:22:43.145706 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:22:43.145706 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:22:43.146880 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:43.146880 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:43.146880 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:43.146880 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:43.146880 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:43.146880 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:43.146880 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:43.146880 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:43.146880 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:43.146880 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:43.146880 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:43.146880 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:43.146880 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:43.146880 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:43.146880 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:43.146880 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:43.146880 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:22:43.148191 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:22:43.148191 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:43.148191 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:43.148191 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 444/902] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:22:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:43.148664 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e81c85-7ab2-472d-b71f-15a168e15e32 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:43.148664 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e81c85-7ab2-472d-b71f-15a168e15e32 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:43.149044 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e81c85-7ab2-472d-b71f-15a168e15e32 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:43.150420 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 445/903] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:22:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:22:43.155305 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e81c85-7ab2-472d-b71f-15a168e15e32 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:43.157898 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 459/904] 10.222.0.47 () {56 vars in 1087 bytes} [Wed Aug 24 06:22:42 2022] GET /identity/v3/services => generated 1497 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:22:43.159290 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c220a819-a72b-435d-b17b-b200664f8f65 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:43.159489 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c220a819-a72b-435d-b17b-b200664f8f65 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:43.159685 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c220a819-a72b-435d-b17b-b200664f8f65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:43.166531 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:43.181125 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-c220a819-a72b-435d-b17b-b200664f8f65 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:22:43.182573 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-c220a819-a72b-435d-b17b-b200664f8f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:22:43.274932 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 460/905] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:22:43 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:22:43.289540 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:43.310523 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e81c85-7ab2-472d-b71f-15a168e15e32 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:43.350829 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716b65b5-be19-4109-b09b-f4da858dda79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:43.352185 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716b65b5-be19-4109-b09b-f4da858dda79 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:43.357598 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716b65b5-be19-4109-b09b-f4da858dda79 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:43.357598 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716b65b5-be19-4109-b09b-f4da858dda79 None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:43.357598 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716b65b5-be19-4109-b09b-f4da858dda79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:43.357598 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716b65b5-be19-4109-b09b-f4da858dda79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:43.357598 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716b65b5-be19-4109-b09b-f4da858dda79 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:43.372306 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 446/906] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 24 06:22:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:43.378194 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c220a819-a72b-435d-b17b-b200664f8f65 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:43.505621 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c220a819-a72b-435d-b17b-b200664f8f65 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:43.550368 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:43.553753 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716b65b5-be19-4109-b09b-f4da858dda79 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:43.559227 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9375f2d-2c51-41f8-8cac-c8cf75c6a410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:43.561789 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9375f2d-2c51-41f8-8cac-c8cf75c6a410 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:43.561789 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9375f2d-2c51-41f8-8cac-c8cf75c6a410 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:43.561789 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9375f2d-2c51-41f8-8cac-c8cf75c6a410 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:43.570966 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9375f2d-2c51-41f8-8cac-c8cf75c6a410 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:43.572922 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9375f2d-2c51-41f8-8cac-c8cf75c6a410 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:43.586382 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-b9375f2d-2c51-41f8-8cac-c8cf75c6a410 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 06:22:43.586382 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:43.586382 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:43.586382 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:43.586382 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:43.586382 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:43.586382 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:43.586382 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:43.586382 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:43.586382 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:43.586382 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:43.586382 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:43.586382 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 06:22:43.586382 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 06:22:43.586382 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 06:22:43.586382 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 06:22:43.588104 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:43.588104 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:43.588104 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 06:22:43.588104 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:22:43.588104 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 06:22:43.588104 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:22:43.588104 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:43.588104 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:43.588104 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:43.588104 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:43.588104 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:43.588104 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:43.588104 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:43.588104 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:43.588104 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:43.588104 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:43.588104 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:43.588104 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:43.589934 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:43.589934 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:43.589934 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 06:22:43.589934 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 06:22:43.589934 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 06:22:43.589934 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 06:22:43.589934 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 06:22:43.589934 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 06:22:43.589934 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 06:22:43.589934 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:43.589934 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 461/907] 10.222.0.47 () {56 vars in 1117 bytes} [Wed Aug 24 06:22:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:43.594211 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9375f2d-2c51-41f8-8cac-c8cf75c6a410 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:43.750030 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9375f2d-2c51-41f8-8cac-c8cf75c6a410 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:43.760413 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716b65b5-be19-4109-b09b-f4da858dda79 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:43.787431 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adbbe9e-5b07-4070-a414-819b967c4e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:43.788198 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adbbe9e-5b07-4070-a414-819b967c4e5e None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:43.789137 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adbbe9e-5b07-4070-a414-819b967c4e5e None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:43.789465 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adbbe9e-5b07-4070-a414-819b967c4e5e None admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:43.790286 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adbbe9e-5b07-4070-a414-819b967c4e5e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:43.790661 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adbbe9e-5b07-4070-a414-819b967c4e5e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:43.792188 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adbbe9e-5b07-4070-a414-819b967c4e5e None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:43.811098 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 462/908] 10.222.0.47 () {56 vars in 1149 bytes} [Wed Aug 24 06:22:43 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:43.821229 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adbbe9e-5b07-4070-a414-819b967c4e5e None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:43.826682 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85828d6b-4989-451d-8833-9de28d619d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:43.827454 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85828d6b-4989-451d-8833-9de28d619d42 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:43.827789 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85828d6b-4989-451d-8833-9de28d619d42 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:43.828065 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85828d6b-4989-451d-8833-9de28d619d42 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:43.828666 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85828d6b-4989-451d-8833-9de28d619d42 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:43.830131 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85828d6b-4989-451d-8833-9de28d619d42 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:43.843508 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-85828d6b-4989-451d-8833-9de28d619d42 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 06:22:43.843508 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:43.843508 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 06:22:43.843508 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 06:22:43.843508 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 06:22:43.843508 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 06:22:43.843508 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 06:22:43.843508 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:22:43.843508 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 06:22:43.843508 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:43.843508 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 06:22:43.843508 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:43.843508 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:43.843508 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:43.843508 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:43.843508 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:43.843508 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:43.846393 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:43.846393 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:43.846393 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:43.846393 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:43.846393 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:43.846393 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:43.846393 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 06:22:43.846393 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 06:22:43.846393 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 06:22:43.846393 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 06:22:43.846393 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:43.846393 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:43.846393 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:43.846393 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:43.846393 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:43.846393 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:43.848032 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:43.848032 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:43.848032 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:43.848032 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:43.848032 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:43.848032 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:43.848032 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:43.848032 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:43.848032 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:43.848032 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:43.848032 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 06:22:43.848032 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:22:43.848032 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 06:22:43.848032 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:43.848032 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 447/909] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Aug 24 06:22:43 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:43.852588 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85828d6b-4989-451d-8833-9de28d619d42 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:43.879574 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:43.890278 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:43.955606 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adbbe9e-5b07-4070-a414-819b967c4e5e None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:44.002156 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace56f25-cd02-45c0-96a4-3d66bb21a40e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:44.003033 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace56f25-cd02-45c0-96a4-3d66bb21a40e None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:44.003465 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace56f25-cd02-45c0-96a4-3d66bb21a40e None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:44.004112 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace56f25-cd02-45c0-96a4-3d66bb21a40e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:44.056421 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85828d6b-4989-451d-8833-9de28d619d42 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:44.092983 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace56f25-cd02-45c0-96a4-3d66bb21a40e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:44.094221 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858c982d-472b-4b2b-b072-abefe5bb8a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:44.094947 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace56f25-cd02-45c0-96a4-3d66bb21a40e None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:44.095218 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858c982d-472b-4b2b-b072-abefe5bb8a7b None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:44.095642 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858c982d-472b-4b2b-b072-abefe5bb8a7b None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:44.096073 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858c982d-472b-4b2b-b072-abefe5bb8a7b None admin] PATH_INFO: `/v3/services` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:44.096847 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858c982d-472b-4b2b-b072-abefe5bb8a7b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:44.097343 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858c982d-472b-4b2b-b072-abefe5bb8a7b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:44.098890 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858c982d-472b-4b2b-b072-abefe5bb8a7b None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:44.109014 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 448/910] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Aug 24 06:22:43 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:44.110215 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858c982d-472b-4b2b-b072-abefe5bb8a7b None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:44.152998 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:44.188576 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:44.215023 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 463/911] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 24 06:22:43 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 06:22:44.223935 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace56f25-cd02-45c0-96a4-3d66bb21a40e None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:44.225086 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858c982d-472b-4b2b-b072-abefe5bb8a7b None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:44.262265 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cda9008-a041-424c-ae0e-728710d39646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:44.263252 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cda9008-a041-424c-ae0e-728710d39646 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:44.263814 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cda9008-a041-424c-ae0e-728710d39646 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:44.264289 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cda9008-a041-424c-ae0e-728710d39646 None admin] PATH_INFO: `/v3/services` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:44.265101 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cda9008-a041-424c-ae0e-728710d39646 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:44.265603 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cda9008-a041-424c-ae0e-728710d39646 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:44.267140 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cda9008-a041-424c-ae0e-728710d39646 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:44.273448 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 449/912] 10.222.0.47 () {56 vars in 1114 bytes} [Wed Aug 24 06:22:44 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:44.281640 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cda9008-a041-424c-ae0e-728710d39646 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:44.362051 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace56f25-cd02-45c0-96a4-3d66bb21a40e None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:44.420643 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73c2810-0697-41b0-b8fa-b0be6be0298a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:44.421243 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73c2810-0697-41b0-b8fa-b0be6be0298a None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:44.421576 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73c2810-0697-41b0-b8fa-b0be6be0298a None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:44.421856 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73c2810-0697-41b0-b8fa-b0be6be0298a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:44.426075 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cda9008-a041-424c-ae0e-728710d39646 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:44.443918 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73c2810-0697-41b0-b8fa-b0be6be0298a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:44.447423 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73c2810-0697-41b0-b8fa-b0be6be0298a None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:44.469251 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-e73c2810-0697-41b0-b8fa-b0be6be0298a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:44.469251 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:44.469251 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:44.469251 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:44.469251 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:44.469251 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:44.469251 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:44.469251 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:44.469251 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:44.469251 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:44.469251 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:44.469251 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:44.469251 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:22:44.469251 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:22:44.469251 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:22:44.469251 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:22:44.470260 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:44.470260 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:44.470260 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:44.470260 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:44.470260 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:44.470260 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:44.470260 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:44.470260 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:44.470260 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:44.470260 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:44.470260 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:44.470260 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:44.470260 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:44.470260 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:44.470260 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:44.470260 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:44.470260 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:22:44.471154 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:22:44.471154 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:44.471154 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:44.473265 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 464/913] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:22:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:44.479780 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73c2810-0697-41b0-b8fa-b0be6be0298a None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:44.480292 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c493ebb8-2de3-4f61-b730-7367165c7f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:44.481036 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c493ebb8-2de3-4f61-b730-7367165c7f1b None admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:44.481353 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c493ebb8-2de3-4f61-b730-7367165c7f1b None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:44.481614 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c493ebb8-2de3-4f61-b730-7367165c7f1b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:44.482281 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c493ebb8-2de3-4f61-b730-7367165c7f1b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:44.483672 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c493ebb8-2de3-4f61-b730-7367165c7f1b None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:44.484590 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:44.492628 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:44.582966 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 450/914] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:22:44 2022] POST /identity/v3/endpoints => generated 346 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:22:44.610111 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73c2810-0697-41b0-b8fa-b0be6be0298a None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:44.678479 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8524336-acce-4d3e-a4f2-9a4fa634392e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:44.680200 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8524336-acce-4d3e-a4f2-9a4fa634392e None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:44.680683 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8524336-acce-4d3e-a4f2-9a4fa634392e None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:44.681138 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8524336-acce-4d3e-a4f2-9a4fa634392e None admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:44.682004 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8524336-acce-4d3e-a4f2-9a4fa634392e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:44.682458 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8524336-acce-4d3e-a4f2-9a4fa634392e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:44.684031 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8524336-acce-4d3e-a4f2-9a4fa634392e None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:44.704183 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 465/915] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 24 06:22:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:44.705957 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c493ebb8-2de3-4f61-b730-7367165c7f1b None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:44.827469 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c493ebb8-2de3-4f61-b730-7367165c7f1b None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:44.868041 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab27c88-0f6a-4c03-8f63-7028475d586a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:44.869449 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab27c88-0f6a-4c03-8f63-7028475d586a None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:44.869923 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab27c88-0f6a-4c03-8f63-7028475d586a None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:44.870473 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab27c88-0f6a-4c03-8f63-7028475d586a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:44.878214 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab27c88-0f6a-4c03-8f63-7028475d586a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:44.879956 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab27c88-0f6a-4c03-8f63-7028475d586a None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:44.892835 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-5ab27c88-0f6a-4c03-8f63-7028475d586a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:22:44.892835 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:44.892835 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:44.892835 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:44.892835 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:44.892835 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:44.892835 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:44.892835 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:44.892835 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:44.892835 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:44.892835 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:44.892835 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:44.892835 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 06:22:44.892835 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 06:22:44.892835 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 06:22:44.893995 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 06:22:44.893995 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:44.893995 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:44.893995 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:44.893995 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:44.893995 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:44.893995 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:44.893995 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:44.893995 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:44.893995 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:44.893995 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:44.893995 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:44.893995 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:44.893995 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:44.893995 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:44.893995 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:44.893995 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:44.895116 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 06:22:44.895116 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 06:22:44.895116 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:22:44.895116 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:22:44.895116 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:22:44.895116 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:22:44.895116 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:22:44.895116 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:44.902689 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 451/916] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 24 06:22:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:44.904319 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8524336-acce-4d3e-a4f2-9a4fa634392e None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:44.974435 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:45.024218 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8524336-acce-4d3e-a4f2-9a4fa634392e None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:45.064929 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95e88ea-78c7-4623-b928-9167cbe9c991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:45.065913 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e88ea-78c7-4623-b928-9167cbe9c991 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:45.066359 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e88ea-78c7-4623-b928-9167cbe9c991 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:45.066785 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e88ea-78c7-4623-b928-9167cbe9c991 None admin] PATH_INFO: `/v3/projects` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:45.067772 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95e88ea-78c7-4623-b928-9167cbe9c991 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:45.068221 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95e88ea-78c7-4623-b928-9167cbe9c991 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:45.070616 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95e88ea-78c7-4623-b928-9167cbe9c991 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:45.089003 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 466/917] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Aug 24 06:22:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:45.091110 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab27c88-0f6a-4c03-8f63-7028475d586a None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:45.153323 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:45.194096 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11ffeb1-bf7b-45ca-9614-660b1c26009b None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:45.195472 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11ffeb1-bf7b-45ca-9614-660b1c26009b None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:45.196256 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11ffeb1-bf7b-45ca-9614-660b1c26009b None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:45.198406 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 467/918] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:22:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:22:45.203232 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7bc971-dc11-4971-b2cb-eec159b8c1dc None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:45.203516 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7bc971-dc11-4971-b2cb-eec159b8c1dc None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:45.203791 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7bc971-dc11-4971-b2cb-eec159b8c1dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:45.218318 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab27c88-0f6a-4c03-8f63-7028475d586a None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:45.225546 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-1e7bc971-dc11-4971-b2cb-eec159b8c1dc None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:22:45.227496 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-1e7bc971-dc11-4971-b2cb-eec159b8c1dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:22:45.263440 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75a3609-5459-472a-9407-3f952e318fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:45.264600 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75a3609-5459-472a-9407-3f952e318fed None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:45.264882 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75a3609-5459-472a-9407-3f952e318fed None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:45.265151 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75a3609-5459-472a-9407-3f952e318fed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:45.266308 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75a3609-5459-472a-9407-3f952e318fed None admin] RBAC: Adding query filter params (role.id=466ed51553a34a62a2c926a8bc552f68, scope.project.id=c02e49a89cdd40d885bfa816f06c58a1, user.id=ea70acd744994250a2905b5dd7163dfb) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:45.266624 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75a3609-5459-472a-9407-3f952e318fed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:45.268891 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75a3609-5459-472a-9407-3f952e318fed None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:45.281297 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 452/919] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 24 06:22:45 2022] GET /identity/v3/role_assignments?user.id=ea70acd744994250a2905b5dd7163dfb&scope.project.id=c02e49a89cdd40d885bfa816f06c58a1&role.id=466ed51553a34a62a2c926a8bc552f68 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:45.315220 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56d9785-83fb-4fbe-8978-8e675ddbe90d None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:45.315504 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56d9785-83fb-4fbe-8978-8e675ddbe90d None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:45.315826 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56d9785-83fb-4fbe-8978-8e675ddbe90d None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:45.317527 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 453/920] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:22:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 06:22:45.323511 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ebfe69-6609-4820-add6-6216a6ae1fc7 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:45.323792 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ebfe69-6609-4820-add6-6216a6ae1fc7 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:45.324074 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ebfe69-6609-4820-add6-6216a6ae1fc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:45.344060 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-12ebfe69-6609-4820-add6-6216a6ae1fc7 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:22:45.344317 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 468/921] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:22:45 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 06:22:45.345388 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-12ebfe69-6609-4820-add6-6216a6ae1fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:22:45.431869 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 454/922] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:22:45 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 06:22:45.633881 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7bc971-dc11-4971-b2cb-eec159b8c1dc None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:45.689497 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:45.701223 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ebfe69-6609-4820-add6-6216a6ae1fc7 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:45.760166 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7bc971-dc11-4971-b2cb-eec159b8c1dc None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:45.766236 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:45.799205 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57ca796-335f-4dce-b6d0-a6d97221ed34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:45.800208 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57ca796-335f-4dce-b6d0-a6d97221ed34 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:45.800694 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57ca796-335f-4dce-b6d0-a6d97221ed34 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:45.801170 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57ca796-335f-4dce-b6d0-a6d97221ed34 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:45.802068 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57ca796-335f-4dce-b6d0-a6d97221ed34 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:45.803667 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57ca796-335f-4dce-b6d0-a6d97221ed34 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:45.814579 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-a57ca796-335f-4dce-b6d0-a6d97221ed34 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 06:22:45.814579 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:45.814579 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 06:22:45.814579 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 06:22:45.814579 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 06:22:45.814579 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 06:22:45.814579 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 06:22:45.814579 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:22:45.814579 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 06:22:45.814579 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:45.814579 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 06:22:45.814579 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:45.814579 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:45.814579 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:45.814579 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:45.814579 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:45.814579 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:45.815622 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:45.815622 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:45.815622 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:45.815622 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:45.815622 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:45.815622 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:45.815622 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 06:22:45.815622 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 06:22:45.815622 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 06:22:45.815622 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 06:22:45.815622 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:45.815622 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:45.815622 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:45.815622 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:45.815622 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:45.815622 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:45.816513 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:45.816513 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:45.816513 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:45.816513 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:45.816513 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:45.816513 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:45.816513 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:45.816513 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:45.816513 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:45.816513 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:45.816513 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 06:22:45.816513 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:22:45.816513 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 06:22:45.816513 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:45.819297 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 469/923] 10.222.0.47 () {56 vars in 1126 bytes} [Wed Aug 24 06:22:45 2022] GET /identity/v3/services/object-store => generated 93 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:45.826405 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57ca796-335f-4dce-b6d0-a6d97221ed34 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:45.834570 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ebfe69-6609-4820-add6-6216a6ae1fc7 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:45.882667 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bdd9bf-3d51-48c0-b7aa-35b0c766e906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:45.883305 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bdd9bf-3d51-48c0-b7aa-35b0c766e906 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:45.883549 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bdd9bf-3d51-48c0-b7aa-35b0c766e906 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:45.883798 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bdd9bf-3d51-48c0-b7aa-35b0c766e906 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:45.884432 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bdd9bf-3d51-48c0-b7aa-35b0c766e906 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:45.886258 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bdd9bf-3d51-48c0-b7aa-35b0c766e906 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:45.897521 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-45bdd9bf-3d51-48c0-b7aa-35b0c766e906 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 06:22:45.897521 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:45.897521 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 06:22:45.897521 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 06:22:45.897521 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 06:22:45.897521 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 06:22:45.897521 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 06:22:45.897521 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:22:45.897521 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 06:22:45.897521 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:45.897521 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 06:22:45.897521 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:45.897521 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:45.897521 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:45.897521 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:45.897521 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:45.898993 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:45.898993 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:45.898993 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:45.898993 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:45.898993 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:45.898993 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:45.898993 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:45.898993 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 06:22:45.898993 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 06:22:45.898993 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 06:22:45.898993 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 06:22:45.898993 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:45.898993 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:45.898993 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:45.898993 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:45.898993 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:45.898993 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:45.899879 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:45.899879 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:45.899879 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:45.899879 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:45.899879 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:45.899879 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:45.899879 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:45.899879 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:45.899879 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:45.899879 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:45.899879 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 06:22:45.899879 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:22:45.899879 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 06:22:45.899879 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:45.899879 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 455/924] 10.222.0.47 () {56 vars in 1132 bytes} [Wed Aug 24 06:22:45 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:22:45.905216 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bdd9bf-3d51-48c0-b7aa-35b0c766e906 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:45.957237 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57ca796-335f-4dce-b6d0-a6d97221ed34 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:45.971870 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:45.994680 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4821d511-55a3-4980-81a6-600ea22ec250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:45.995295 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4821d511-55a3-4980-81a6-600ea22ec250 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:45.995574 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4821d511-55a3-4980-81a6-600ea22ec250 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:45.995846 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4821d511-55a3-4980-81a6-600ea22ec250 None admin] PATH_INFO: `/v3/services` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:45.996537 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4821d511-55a3-4980-81a6-600ea22ec250 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:45.996842 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4821d511-55a3-4980-81a6-600ea22ec250 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:46.000205 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4821d511-55a3-4980-81a6-600ea22ec250 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:46.010073 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 470/925] 10.222.0.47 () {56 vars in 1122 bytes} [Wed Aug 24 06:22:45 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:22:46.011610 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4821d511-55a3-4980-81a6-600ea22ec250 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:46.022159 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bdd9bf-3d51-48c0-b7aa-35b0c766e906 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:46.034586 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:46.060903 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa994ed-c2f7-4554-b96d-65875ff7216e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:46.061867 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa994ed-c2f7-4554-b96d-65875ff7216e None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:46.062623 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa994ed-c2f7-4554-b96d-65875ff7216e None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:46.063100 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa994ed-c2f7-4554-b96d-65875ff7216e None admin] PATH_INFO: `/v3/services` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:46.064239 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa994ed-c2f7-4554-b96d-65875ff7216e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:46.065000 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa994ed-c2f7-4554-b96d-65875ff7216e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:46.066465 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa994ed-c2f7-4554-b96d-65875ff7216e None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:46.072864 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 456/926] 10.222.0.47 () {56 vars in 1126 bytes} [Wed Aug 24 06:22:45 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:46.078136 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa994ed-c2f7-4554-b96d-65875ff7216e None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:46.134418 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4821d511-55a3-4980-81a6-600ea22ec250 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:46.174407 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dc3c3c-24cc-4a6e-b0a0-d0e5169e1f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:46.175020 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dc3c3c-24cc-4a6e-b0a0-d0e5169e1f4c None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:46.175297 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dc3c3c-24cc-4a6e-b0a0-d0e5169e1f4c None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:46.175569 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dc3c3c-24cc-4a6e-b0a0-d0e5169e1f4c None admin] PATH_INFO: `/v3/services` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:46.176200 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dc3c3c-24cc-4a6e-b0a0-d0e5169e1f4c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:46.176499 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dc3c3c-24cc-4a6e-b0a0-d0e5169e1f4c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:46.177918 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dc3c3c-24cc-4a6e-b0a0-d0e5169e1f4c None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:46.183620 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 471/927] 10.222.0.47 () {56 vars in 1122 bytes} [Wed Aug 24 06:22:46 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:46.189568 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dc3c3c-24cc-4a6e-b0a0-d0e5169e1f4c None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:46.221002 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa994ed-c2f7-4554-b96d-65875ff7216e None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:46.249281 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:46.268201 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3a8e85-c9f5-40f0-b013-758a68ab246a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:46.268823 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3a8e85-c9f5-40f0-b013-758a68ab246a None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:46.269105 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3a8e85-c9f5-40f0-b013-758a68ab246a None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:46.269407 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3a8e85-c9f5-40f0-b013-758a68ab246a None admin] PATH_INFO: `/v3/services` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:46.274206 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3a8e85-c9f5-40f0-b013-758a68ab246a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:46.274206 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3a8e85-c9f5-40f0-b013-758a68ab246a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:46.274206 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3a8e85-c9f5-40f0-b013-758a68ab246a None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:46.281312 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 457/928] 10.222.0.47 () {56 vars in 1126 bytes} [Wed Aug 24 06:22:46 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:46.289836 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3a8e85-c9f5-40f0-b013-758a68ab246a None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:46.333939 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dc3c3c-24cc-4a6e-b0a0-d0e5169e1f4c None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:46.345612 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:46.383985 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a9c01a-cf2a-425e-862b-f6acdbfa65d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:46.384904 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a9c01a-cf2a-425e-862b-f6acdbfa65d6 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:46.385286 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a9c01a-cf2a-425e-862b-f6acdbfa65d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:46.385875 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a9c01a-cf2a-425e-862b-f6acdbfa65d6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:46.387286 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a9c01a-cf2a-425e-862b-f6acdbfa65d6 None admin] RBAC: Adding query filter params (interface=public, service_id=7399165cb4f346abb275646eb6bd45e8, region_id=RegionOne) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:46.387965 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a9c01a-cf2a-425e-862b-f6acdbfa65d6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:46.390530 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a9c01a-cf2a-425e-862b-f6acdbfa65d6 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:46.405223 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 472/929] 10.222.0.47 () {56 vars in 1251 bytes} [Wed Aug 24 06:22:46 2022] GET /identity/v3/endpoints?service_id=7399165cb4f346abb275646eb6bd45e8&interface=public®ion_id=RegionOne => generated 194 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:22:46.406333 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a9c01a-cf2a-425e-862b-f6acdbfa65d6 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:46.417591 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3a8e85-c9f5-40f0-b013-758a68ab246a None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:46.465563 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d9a0f9-0717-441b-940a-b72cf7203b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:46.466189 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d9a0f9-0717-441b-940a-b72cf7203b7f None admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:46.466468 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d9a0f9-0717-441b-940a-b72cf7203b7f None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:46.466894 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d9a0f9-0717-441b-940a-b72cf7203b7f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:46.467683 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d9a0f9-0717-441b-940a-b72cf7203b7f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:46.469401 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d9a0f9-0717-441b-940a-b72cf7203b7f None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:46.539570 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a9c01a-cf2a-425e-862b-f6acdbfa65d6 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:46.543035 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:46.572465 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 458/930] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:22:46 2022] POST /identity/v3/endpoints => generated 347 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:22:46.600722 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13427473-2c33-4768-b688-22817be75e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:46.601707 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13427473-2c33-4768-b688-22817be75e7b None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:46.602181 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13427473-2c33-4768-b688-22817be75e7b None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:46.602462 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13427473-2c33-4768-b688-22817be75e7b None admin] PATH_INFO: `/v3/services` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:46.603094 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13427473-2c33-4768-b688-22817be75e7b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:46.604643 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13427473-2c33-4768-b688-22817be75e7b None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:46.617912 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 473/931] 10.222.0.47 () {56 vars in 1087 bytes} [Wed Aug 24 06:22:46 2022] GET /identity/v3/services => generated 1497 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:22:46.776885 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e9ffd1-05dd-44a1-af9b-6a4198d553e2 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:46.777078 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e9ffd1-05dd-44a1-af9b-6a4198d553e2 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:46.777413 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e9ffd1-05dd-44a1-af9b-6a4198d553e2 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:46.779140 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 459/932] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:22:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:22:46.784994 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d43898-76cf-4f3f-a7b1-81f756901a50 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:46.785325 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d43898-76cf-4f3f-a7b1-81f756901a50 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:46.785602 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d43898-76cf-4f3f-a7b1-81f756901a50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:46.806756 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-79d43898-76cf-4f3f-a7b1-81f756901a50 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:22:46.807973 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-79d43898-76cf-4f3f-a7b1-81f756901a50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:22:46.887237 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 474/933] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:22:46 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:22:47.165747 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e9ffd1-05dd-44a1-af9b-6a4198d553e2 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:47.271783 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:47.283610 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e9ffd1-05dd-44a1-af9b-6a4198d553e2 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:47.305215 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51e2e7d-e936-478b-aff6-b0cd5fa736f6 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:47.305738 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51e2e7d-e936-478b-aff6-b0cd5fa736f6 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:47.305817 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51e2e7d-e936-478b-aff6-b0cd5fa736f6 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:47.309267 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 475/934] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:22:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 06:22:47.313621 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10842431-4355-4aad-8314-5cd1adc27120 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:47.313904 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10842431-4355-4aad-8314-5cd1adc27120 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:47.314179 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10842431-4355-4aad-8314-5cd1adc27120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:47.328245 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe209e3-620f-4f7e-833f-dc2f3d554056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:47.329817 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe209e3-620f-4f7e-833f-dc2f3d554056 None admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:47.330434 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe209e3-620f-4f7e-833f-dc2f3d554056 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:47.330946 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe209e3-620f-4f7e-833f-dc2f3d554056 None admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:47.331811 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe209e3-620f-4f7e-833f-dc2f3d554056 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:47.333573 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe209e3-620f-4f7e-833f-dc2f3d554056 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:47.334693 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-10842431-4355-4aad-8314-5cd1adc27120 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:22:47.336754 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-10842431-4355-4aad-8314-5cd1adc27120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:22:47.337832 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: WARNING keystone.server.flask.common [None req-bfe209e3-620f-4f7e-833f-dc2f3d554056 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 24 06:22:47.403961 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.sql.core [None req-bfe209e3-620f-4f7e-833f-dc2f3d554056 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 06:22:47.403961 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 24 06:22:47.403961 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [parameters: {'id': '2de3484b947b40eab4a63bfdbd75c9d2', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 24 06:22:47.403961 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81724) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 06:22:47.411868 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-bfe209e3-620f-4f7e-833f-dc2f3d554056 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 2de3484b947b40eab4a63bfdbd75c9d2.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 2de3484b947b40eab4a63bfdbd75c9d2. Aug 24 06:22:47.411868 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:47.411868 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 06:22:47.411868 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 06:22:47.411868 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 06:22:47.411868 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 06:22:47.411868 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 06:22:47.411868 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 06:22:47.411868 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 06:22:47.411868 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application conn.query(q) Aug 24 06:22:47.411868 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 06:22:47.411868 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 06:22:47.411868 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 06:22:47.412710 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application result.read() Aug 24 06:22:47.412710 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 06:22:47.412710 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 06:22:47.412710 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 06:22:47.412710 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 06:22:47.412710 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 06:22:47.412710 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 06:22:47.412710 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 06:22:47.412710 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 06:22:47.412710 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 06:22:47.412710 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:47.412710 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 06:22:47.412710 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:47.412710 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:47.412710 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 06:22:47.412710 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 06:22:47.412710 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 24 06:22:47.412710 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 24 06:22:47.413891 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 06:22:47.413891 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application next(self.gen) Aug 24 06:22:47.413891 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 06:22:47.413891 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application yield resource Aug 24 06:22:47.413891 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 06:22:47.413891 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application next(self.gen) Aug 24 06:22:47.413891 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 06:22:47.413891 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 06:22:47.413891 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 06:22:47.413891 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 06:22:47.413891 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 06:22:47.413891 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise self.value Aug 24 06:22:47.413891 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 06:22:47.413891 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 06:22:47.413891 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 06:22:47.413891 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application session.commit() Aug 24 06:22:47.413891 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 06:22:47.414914 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 06:22:47.414914 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 06:22:47.414914 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 06:22:47.414914 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 06:22:47.414914 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application self.session.flush() Aug 24 06:22:47.414914 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 06:22:47.414914 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 06:22:47.414914 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 06:22:47.414914 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 06:22:47.414914 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 06:22:47.414914 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application compat.raise_( Aug 24 06:22:47.414914 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 06:22:47.414914 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception Aug 24 06:22:47.414914 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 06:22:47.414914 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 06:22:47.414914 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 06:22:47.414914 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 06:22:47.415956 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 06:22:47.415956 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 06:22:47.415956 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 06:22:47.415956 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 06:22:47.415956 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 24 06:22:47.415956 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 24 06:22:47.415956 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 06:22:47.415956 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 06:22:47.415956 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 06:22:47.415956 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 06:22:47.415956 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 06:22:47.415956 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 06:22:47.415956 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 06:22:47.415956 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 06:22:47.415956 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 06:22:47.416860 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 06:22:47.416860 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 06:22:47.416860 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception Aug 24 06:22:47.416860 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 06:22:47.416860 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 06:22:47.416860 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 06:22:47.416860 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 06:22:47.416860 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 06:22:47.416860 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 06:22:47.416860 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 06:22:47.416860 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application conn.query(q) Aug 24 06:22:47.416860 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 06:22:47.416860 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 06:22:47.416860 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 06:22:47.416860 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application result.read() Aug 24 06:22:47.416860 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 06:22:47.416860 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 06:22:47.420245 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 06:22:47.420245 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 06:22:47.420245 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 06:22:47.420245 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 06:22:47.420245 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 06:22:47.420245 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 06:22:47.420245 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 06:22:47.420245 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 24 06:22:47.420245 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [parameters: {'id': '2de3484b947b40eab4a63bfdbd75c9d2', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 24 06:22:47.420245 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 06:22:47.420245 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:47.420245 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 06:22:47.420245 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:47.420245 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:47.420245 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 24 06:22:47.422346 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 24 06:22:47.422346 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 06:22:47.422346 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 06:22:47.422346 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 24 06:22:47.422346 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:47.422346 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 06:22:47.422346 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:47.422346 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:47.422346 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:47.422346 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:47.422346 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:47.422346 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:47.422346 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:47.422346 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:47.422346 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:47.422346 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:47.422346 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:47.425241 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:47.425241 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 24 06:22:47.425241 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 24 06:22:47.425241 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:47.425241 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:47.425241 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 24 06:22:47.425241 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 24 06:22:47.425241 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 2de3484b947b40eab4a63bfdbd75c9d2. Aug 24 06:22:47.425241 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:47.425241 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 460/935] 10.222.0.47 () {60 vars in 1140 bytes} [Wed Aug 24 06:22:47 2022] POST /identity/v3/projects => generated 262 bytes in 249 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:47.425241 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe209e3-620f-4f7e-833f-dc2f3d554056 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:47.437822 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 476/936] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:22:47 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 06:22:47.548027 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe209e3-620f-4f7e-833f-dc2f3d554056 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:47.585851 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356b496c-25d6-4f38-8d86-d794c3ac6062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:47.586669 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356b496c-25d6-4f38-8d86-d794c3ac6062 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:47.587050 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356b496c-25d6-4f38-8d86-d794c3ac6062 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:47.587419 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356b496c-25d6-4f38-8d86-d794c3ac6062 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:47.595069 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356b496c-25d6-4f38-8d86-d794c3ac6062 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:47.596732 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356b496c-25d6-4f38-8d86-d794c3ac6062 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:47.609155 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-356b496c-25d6-4f38-8d86-d794c3ac6062 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:22:47.609155 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:47.609155 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:47.609155 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:47.609155 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:47.609155 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:47.609155 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:47.609155 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:47.609155 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:47.609155 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:47.609155 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:47.609155 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:47.609155 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 06:22:47.609155 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 06:22:47.609155 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 06:22:47.610119 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 06:22:47.610119 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:47.610119 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:47.610119 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:47.610119 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:47.610119 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:47.610119 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:47.610119 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:47.610119 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:47.610119 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:47.610119 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:47.610119 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:47.610119 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:47.610119 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:47.610119 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:47.610119 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:47.610119 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:47.611032 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 06:22:47.611032 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 06:22:47.611032 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:22:47.611032 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:22:47.611032 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:22:47.611032 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:22:47.611032 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:22:47.611032 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:47.611032 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 461/937] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 24 06:22:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:47.616548 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10842431-4355-4aad-8314-5cd1adc27120 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:47.673324 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:47.751399 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356b496c-25d6-4f38-8d86-d794c3ac6062 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:47.755976 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10842431-4355-4aad-8314-5cd1adc27120 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:47.766314 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:47.793819 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923bb20f-51f8-4e08-a863-2deadae27cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:47.794491 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923bb20f-51f8-4e08-a863-2deadae27cf0 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:47.794730 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923bb20f-51f8-4e08-a863-2deadae27cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:47.795007 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923bb20f-51f8-4e08-a863-2deadae27cf0 None admin] PATH_INFO: `/v3/projects` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:47.795687 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923bb20f-51f8-4e08-a863-2deadae27cf0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:47.795991 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923bb20f-51f8-4e08-a863-2deadae27cf0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:47.797475 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923bb20f-51f8-4e08-a863-2deadae27cf0 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:47.816815 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 477/938] 10.222.0.47 () {56 vars in 1112 bytes} [Wed Aug 24 06:22:47 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:47.890849 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356b496c-25d6-4f38-8d86-d794c3ac6062 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:47.935729 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3fbfea-db07-456f-8218-01c332421536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:47.936669 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3fbfea-db07-456f-8218-01c332421536 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:47.937106 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3fbfea-db07-456f-8218-01c332421536 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:47.937562 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3fbfea-db07-456f-8218-01c332421536 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:47.951377 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3fbfea-db07-456f-8218-01c332421536 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:22:47.957267 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-8a3fbfea-db07-456f-8218-01c332421536 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 06:22:47.957267 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:47.957267 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:47.957267 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:47.957267 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:47.957267 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:47.957267 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:47.957267 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:47.957267 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:47.957267 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:47.957267 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:47.957267 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:47.957267 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:22:47.957267 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:22:47.957267 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:22:47.957267 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:22:47.958471 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:22:47.958471 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:22:47.958471 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:22:47.958471 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:22:47.958471 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:47.958471 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:47.958471 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:47.958471 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:47.958471 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:47.958471 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:47.958471 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:47.958471 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:47.958471 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:47.958471 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:47.958471 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:47.958471 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:47.959514 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:47.959514 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:47.959514 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:47.959514 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:47.959514 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:22:47.959514 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:22:47.959514 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:22:47.959514 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:22:47.959514 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:22:47.959514 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:22:47.959514 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 06:22:47.959514 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:47.969296 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 462/939] 10.222.0.47 () {56 vars in 1120 bytes} [Wed Aug 24 06:22:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:22:47.971647 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923bb20f-51f8-4e08-a863-2deadae27cf0 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:48.081736 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:48.102719 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923bb20f-51f8-4e08-a863-2deadae27cf0 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:48.140564 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c564fd-c8d7-4bfb-80cc-1cb91499ee63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:48.141579 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c564fd-c8d7-4bfb-80cc-1cb91499ee63 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:48.142063 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c564fd-c8d7-4bfb-80cc-1cb91499ee63 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:48.142533 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c564fd-c8d7-4bfb-80cc-1cb91499ee63 None admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:48.143422 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c564fd-c8d7-4bfb-80cc-1cb91499ee63 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:48.143920 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c564fd-c8d7-4bfb-80cc-1cb91499ee63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:48.145810 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c564fd-c8d7-4bfb-80cc-1cb91499ee63 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:48.156062 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 478/940] 10.222.0.47 () {56 vars in 1115 bytes} [Wed Aug 24 06:22:47 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:48.161589 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3fbfea-db07-456f-8218-01c332421536 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:48.245356 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:48.279946 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3fbfea-db07-456f-8218-01c332421536 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:48.319808 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf1d399-f8b7-431f-a840-ffde616bedfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:48.320804 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf1d399-f8b7-431f-a840-ffde616bedfa None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:48.321377 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf1d399-f8b7-431f-a840-ffde616bedfa None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:48.321791 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf1d399-f8b7-431f-a840-ffde616bedfa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:48.335701 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf1d399-f8b7-431f-a840-ffde616bedfa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:48.337169 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf1d399-f8b7-431f-a840-ffde616bedfa None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:48.353545 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-acf1d399-f8b7-431f-a840-ffde616bedfa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:48.353545 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:48.353545 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:48.353545 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:48.353545 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:48.353545 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:48.353545 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:48.353545 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:48.353545 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:48.353545 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:48.353545 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:48.353545 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:48.353545 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:22:48.353545 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:22:48.353545 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:22:48.353545 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:22:48.354564 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:48.354564 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:48.354564 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:48.354564 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:48.354564 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:48.354564 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:48.354564 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:48.354564 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:48.354564 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:48.354564 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:48.354564 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:48.354564 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:48.354564 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:48.354564 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:48.354564 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:48.354564 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:48.354564 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:22:48.355447 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:22:48.355447 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:48.355447 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:48.359493 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 463/941] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:22:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:48.360925 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c564fd-c8d7-4bfb-80cc-1cb91499ee63 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:48.481991 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c564fd-c8d7-4bfb-80cc-1cb91499ee63 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:48.518273 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2bfb92-bd2e-4cf7-af23-e455463c62e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:48.519282 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2bfb92-bd2e-4cf7-af23-e455463c62e4 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:48.519792 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2bfb92-bd2e-4cf7-af23-e455463c62e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:48.520271 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2bfb92-bd2e-4cf7-af23-e455463c62e4 None admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:48.521504 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2bfb92-bd2e-4cf7-af23-e455463c62e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:48.522018 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2bfb92-bd2e-4cf7-af23-e455463c62e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:48.524156 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2bfb92-bd2e-4cf7-af23-e455463c62e4 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:48.537810 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 479/942] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 24 06:22:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:48.547168 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf1d399-f8b7-431f-a840-ffde616bedfa None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:48.663729 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf1d399-f8b7-431f-a840-ffde616bedfa None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:48.688065 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:48.700564 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd5aa78-50fb-46c1-9c2d-f9a5f1d3dafc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:48.707296 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd5aa78-50fb-46c1-9c2d-f9a5f1d3dafc None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:48.707296 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd5aa78-50fb-46c1-9c2d-f9a5f1d3dafc None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:48.707296 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd5aa78-50fb-46c1-9c2d-f9a5f1d3dafc None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:48.710147 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd5aa78-50fb-46c1-9c2d-f9a5f1d3dafc None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:48.711590 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd5aa78-50fb-46c1-9c2d-f9a5f1d3dafc None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:48.723606 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-2bd5aa78-50fb-46c1-9c2d-f9a5f1d3dafc None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 06:22:48.723606 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:48.723606 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:48.723606 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:48.723606 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:48.723606 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:48.723606 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:48.723606 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:48.723606 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:48.723606 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:48.723606 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:48.723606 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:48.723606 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 06:22:48.723606 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 06:22:48.723606 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 06:22:48.723606 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 06:22:48.724652 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:48.724652 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:48.724652 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 06:22:48.724652 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:22:48.724652 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 06:22:48.724652 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:22:48.724652 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:48.724652 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:48.724652 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:48.724652 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:48.724652 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:48.724652 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:48.724652 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:48.724652 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:48.724652 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:48.724652 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:48.724652 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:48.724652 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:48.726258 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:48.726258 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:48.726258 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 06:22:48.726258 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 06:22:48.726258 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 06:22:48.726258 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 06:22:48.726258 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 06:22:48.726258 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 06:22:48.726258 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 06:22:48.726258 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:48.729235 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 464/943] 10.222.0.47 () {56 vars in 1117 bytes} [Wed Aug 24 06:22:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:48.735010 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2bfb92-bd2e-4cf7-af23-e455463c62e4 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:48.747054 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:48.752199 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256d0a21-dfab-4223-ac92-3434151d102c None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:48.752684 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256d0a21-dfab-4223-ac92-3434151d102c None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:48.753107 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256d0a21-dfab-4223-ac92-3434151d102c None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:48.757766 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 465/944] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:22:48 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 06:22:48.764262 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3d5f77-f8d2-4b67-b474-a0b75042c876 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:48.764561 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3d5f77-f8d2-4b67-b474-a0b75042c876 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:48.764827 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3d5f77-f8d2-4b67-b474-a0b75042c876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:48.793108 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-4d3d5f77-f8d2-4b67-b474-a0b75042c876 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:22:48.794336 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-4d3d5f77-f8d2-4b67-b474-a0b75042c876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:22:48.874406 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 466/945] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:22:48 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 06:22:48.877440 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2bfb92-bd2e-4cf7-af23-e455463c62e4 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:48.883085 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11119379-989f-4f31-8fbd-eac687d62cbe None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:48.883376 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11119379-989f-4f31-8fbd-eac687d62cbe None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:48.883818 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11119379-989f-4f31-8fbd-eac687d62cbe None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:48.885348 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 467/946] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:22:48 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:22:48.894916 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a843837-002e-415f-84dd-d56c5df651b5 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:48.895199 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a843837-002e-415f-84dd-d56c5df651b5 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:48.895998 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a843837-002e-415f-84dd-d56c5df651b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:48.921523 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-2a843837-002e-415f-84dd-d56c5df651b5 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:22:48.922748 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0f32b1-ae18-4a04-ad11-1324af16fd6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:48.922910 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-2a843837-002e-415f-84dd-d56c5df651b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:22:48.923367 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0f32b1-ae18-4a04-ad11-1324af16fd6c None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:48.923648 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0f32b1-ae18-4a04-ad11-1324af16fd6c None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:48.923963 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0f32b1-ae18-4a04-ad11-1324af16fd6c None admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:48.924971 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0f32b1-ae18-4a04-ad11-1324af16fd6c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:48.925323 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0f32b1-ae18-4a04-ad11-1324af16fd6c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:48.926918 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0f32b1-ae18-4a04-ad11-1324af16fd6c None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:48.952491 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 480/947] 10.222.0.47 () {56 vars in 1149 bytes} [Wed Aug 24 06:22:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:48.954047 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0f32b1-ae18-4a04-ad11-1324af16fd6c None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:49.017353 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 468/948] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:22:48 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:22:49.053885 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:49.086973 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0f32b1-ae18-4a04-ad11-1324af16fd6c None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:49.125094 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce139648-6b9f-4293-95ba-95b10dd55c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:49.125776 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce139648-6b9f-4293-95ba-95b10dd55c33 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:49.126058 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce139648-6b9f-4293-95ba-95b10dd55c33 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:49.126332 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce139648-6b9f-4293-95ba-95b10dd55c33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:49.137700 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce139648-6b9f-4293-95ba-95b10dd55c33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:49.139169 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce139648-6b9f-4293-95ba-95b10dd55c33 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:49.155991 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-ce139648-6b9f-4293-95ba-95b10dd55c33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:49.155991 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:49.155991 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:49.155991 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:49.155991 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:49.155991 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:49.155991 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:49.155991 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:49.155991 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:49.155991 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:49.155991 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:49.155991 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:49.155991 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:22:49.155991 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:22:49.155991 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:22:49.155991 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:22:49.157005 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:49.157005 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:49.157005 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:49.157005 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:49.157005 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:49.157005 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:49.157005 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:49.157005 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:49.157005 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:49.157005 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:49.157005 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:49.157005 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:49.157005 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:49.157005 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:49.157005 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:49.157005 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:49.157005 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:22:49.157942 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:22:49.157942 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:49.157942 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:49.157942 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 481/949] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:22:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:22:49.164074 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a843837-002e-415f-84dd-d56c5df651b5 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:49.178937 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce139648-6b9f-4293-95ba-95b10dd55c33 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:49.224539 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:49.304533 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a843837-002e-415f-84dd-d56c5df651b5 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:49.306866 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce139648-6b9f-4293-95ba-95b10dd55c33 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:49.335042 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:49.350273 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78cc8fd-0462-4771-baf9-e3ddf5524992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:49.351589 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78cc8fd-0462-4771-baf9-e3ddf5524992 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:49.351935 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78cc8fd-0462-4771-baf9-e3ddf5524992 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:49.352288 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78cc8fd-0462-4771-baf9-e3ddf5524992 None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:49.353029 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78cc8fd-0462-4771-baf9-e3ddf5524992 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:49.353487 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78cc8fd-0462-4771-baf9-e3ddf5524992 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:49.354895 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78cc8fd-0462-4771-baf9-e3ddf5524992 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:49.356760 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd901fe7-c027-4963-a964-563a07aabfb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:49.357508 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd901fe7-c027-4963-a964-563a07aabfb1 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:49.357860 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd901fe7-c027-4963-a964-563a07aabfb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:49.358201 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd901fe7-c027-4963-a964-563a07aabfb1 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:49.358866 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd901fe7-c027-4963-a964-563a07aabfb1 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:49.360302 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd901fe7-c027-4963-a964-563a07aabfb1 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:49.372417 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 469/950] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 24 06:22:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:49.372417 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78cc8fd-0462-4771-baf9-e3ddf5524992 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:49.372714 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-fd901fe7-c027-4963-a964-563a07aabfb1 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 06:22:49.372714 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:49.372714 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 06:22:49.372714 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 06:22:49.372714 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 06:22:49.372714 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 06:22:49.372714 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 06:22:49.372714 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:22:49.372714 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 06:22:49.372714 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:49.372714 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 06:22:49.372714 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:49.372714 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:49.372714 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:49.372714 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:49.372714 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:49.372714 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:49.374891 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:49.374891 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:49.374891 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:49.374891 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:49.374891 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:49.374891 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:49.374891 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 06:22:49.374891 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 06:22:49.374891 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 06:22:49.374891 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 06:22:49.374891 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:49.374891 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:49.374891 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:49.374891 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:49.374891 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:49.374891 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:49.375727 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:49.375727 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:49.375727 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:49.375727 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:49.375727 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:49.375727 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:49.375727 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:49.375727 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:49.375727 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:49.375727 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:49.375727 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 06:22:49.375727 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:22:49.375727 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 06:22:49.375727 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:49.375727 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 482/951] 10.222.0.47 () {56 vars in 1126 bytes} [Wed Aug 24 06:22:49 2022] GET /identity/v3/services/object-store => generated 93 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:22:49.378836 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd901fe7-c027-4963-a964-563a07aabfb1 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:49.495042 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78cc8fd-0462-4771-baf9-e3ddf5524992 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:49.496670 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:49.506431 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd901fe7-c027-4963-a964-563a07aabfb1 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:49.546320 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60680da5-c6bd-4c5d-9ee2-535b5a3fe4cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:49.547296 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60680da5-c6bd-4c5d-9ee2-535b5a3fe4cf None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:49.547735 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60680da5-c6bd-4c5d-9ee2-535b5a3fe4cf None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:49.548008 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60680da5-c6bd-4c5d-9ee2-535b5a3fe4cf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:49.551117 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dbbdc9-aaf6-4929-9709-3623fba3e547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:49.551770 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dbbdc9-aaf6-4929-9709-3623fba3e547 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:49.552047 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dbbdc9-aaf6-4929-9709-3623fba3e547 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:49.552323 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dbbdc9-aaf6-4929-9709-3623fba3e547 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:49.552914 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dbbdc9-aaf6-4929-9709-3623fba3e547 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:49.555202 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dbbdc9-aaf6-4929-9709-3623fba3e547 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:49.558435 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60680da5-c6bd-4c5d-9ee2-535b5a3fe4cf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:49.559889 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60680da5-c6bd-4c5d-9ee2-535b5a3fe4cf None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:49.564371 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-23dbbdc9-aaf6-4929-9709-3623fba3e547 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 06:22:49.564371 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:49.564371 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 06:22:49.564371 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 06:22:49.564371 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 06:22:49.564371 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 06:22:49.564371 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 06:22:49.564371 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:22:49.564371 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 06:22:49.564371 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:49.564371 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 06:22:49.564371 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:49.564371 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:49.564371 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:49.564371 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:49.564371 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:49.564371 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:49.565537 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:49.565537 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:49.565537 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:49.565537 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:49.565537 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:49.565537 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:49.565537 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 06:22:49.565537 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 06:22:49.565537 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 06:22:49.565537 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 06:22:49.565537 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:49.565537 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:49.565537 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:49.565537 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:49.565537 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:49.565537 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:49.566619 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:49.566619 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:49.566619 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:49.566619 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:49.566619 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:49.566619 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:49.566619 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:49.566619 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:49.566619 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:49.566619 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:49.566619 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 06:22:49.566619 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:22:49.566619 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 06:22:49.566619 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:49.570730 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 470/952] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 24 06:22:49 2022] GET /identity/v3/services/compute => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:49.571459 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dbbdc9-aaf6-4929-9709-3623fba3e547 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:49.575099 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-60680da5-c6bd-4c5d-9ee2-535b5a3fe4cf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:22:49.575099 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:49.575099 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:49.575099 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:49.575099 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:49.575099 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:49.575099 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:49.575099 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:49.575099 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:49.575099 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:49.575099 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:49.575099 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:49.575099 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 06:22:49.575099 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 06:22:49.575099 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 06:22:49.576215 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 06:22:49.576215 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:49.576215 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:49.576215 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:49.576215 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:49.576215 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:49.576215 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:49.576215 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:49.576215 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:49.576215 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:49.576215 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:49.576215 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:49.576215 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:49.576215 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:49.576215 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:49.576215 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:49.576215 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:49.577349 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 06:22:49.577349 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 06:22:49.577349 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:22:49.577349 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:22:49.577349 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:22:49.577349 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:22:49.577349 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:22:49.577349 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:49.581349 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 483/953] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 24 06:22:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:49.584177 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60680da5-c6bd-4c5d-9ee2-535b5a3fe4cf None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:49.630206 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:49.704375 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60680da5-c6bd-4c5d-9ee2-535b5a3fe4cf None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:49.715293 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dbbdc9-aaf6-4929-9709-3623fba3e547 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:49.741120 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb22f376-95c6-424c-ba6f-85d4ce4395c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:49.741990 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb22f376-95c6-424c-ba6f-85d4ce4395c8 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:49.742183 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb22f376-95c6-424c-ba6f-85d4ce4395c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:49.742546 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb22f376-95c6-424c-ba6f-85d4ce4395c8 None admin] PATH_INFO: `/v3/services` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:49.743179 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb22f376-95c6-424c-ba6f-85d4ce4395c8 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:49.743482 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb22f376-95c6-424c-ba6f-85d4ce4395c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:49.744861 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb22f376-95c6-424c-ba6f-85d4ce4395c8 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:49.749917 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 484/954] 10.222.0.47 () {56 vars in 1112 bytes} [Wed Aug 24 06:22:49 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:49.750773 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb22f376-95c6-424c-ba6f-85d4ce4395c8 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:49.760410 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef59c717-37a7-4246-97f9-27066f7f6a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:49.761506 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef59c717-37a7-4246-97f9-27066f7f6a39 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:49.761889 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef59c717-37a7-4246-97f9-27066f7f6a39 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:49.762258 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef59c717-37a7-4246-97f9-27066f7f6a39 None admin] PATH_INFO: `/v3/services` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:49.763057 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef59c717-37a7-4246-97f9-27066f7f6a39 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:49.763458 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef59c717-37a7-4246-97f9-27066f7f6a39 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:49.765020 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef59c717-37a7-4246-97f9-27066f7f6a39 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:49.770936 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 471/955] 10.222.0.47 () {56 vars in 1122 bytes} [Wed Aug 24 06:22:49 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:49.771589 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef59c717-37a7-4246-97f9-27066f7f6a39 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:49.806494 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:49.870650 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb22f376-95c6-424c-ba6f-85d4ce4395c8 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:49.874203 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:49.891669 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef59c717-37a7-4246-97f9-27066f7f6a39 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:49.916910 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b533e95e-049f-4dce-a68f-d6e5b256e000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:49.917796 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b533e95e-049f-4dce-a68f-d6e5b256e000 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:49.918429 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b533e95e-049f-4dce-a68f-d6e5b256e000 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:49.918718 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b533e95e-049f-4dce-a68f-d6e5b256e000 None admin] PATH_INFO: `/v3/projects` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:49.919417 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b533e95e-049f-4dce-a68f-d6e5b256e000 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:49.919720 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b533e95e-049f-4dce-a68f-d6e5b256e000 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:49.922780 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b533e95e-049f-4dce-a68f-d6e5b256e000 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:49.933768 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 485/956] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Aug 24 06:22:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:49.934703 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b533e95e-049f-4dce-a68f-d6e5b256e000 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:49.940568 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dd0e3c-a19a-4c83-a133-5953ad408fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:49.941517 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dd0e3c-a19a-4c83-a133-5953ad408fbb None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:49.941919 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dd0e3c-a19a-4c83-a133-5953ad408fbb None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:49.942372 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dd0e3c-a19a-4c83-a133-5953ad408fbb None admin] PATH_INFO: `/v3/services` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:49.943343 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dd0e3c-a19a-4c83-a133-5953ad408fbb None admin] RBAC: Adding query filter params (type=compute) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:49.943777 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dd0e3c-a19a-4c83-a133-5953ad408fbb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:49.946361 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dd0e3c-a19a-4c83-a133-5953ad408fbb None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:49.957230 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 472/957] 10.222.0.47 () {56 vars in 1112 bytes} [Wed Aug 24 06:22:49 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:49.957230 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940d37e5-74ab-4e6c-b7ee-9124af131dfc None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:49.957230 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940d37e5-74ab-4e6c-b7ee-9124af131dfc None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:49.957230 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940d37e5-74ab-4e6c-b7ee-9124af131dfc None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:49.964465 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 473/958] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:22:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:22:49.964465 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940d37e5-74ab-4e6c-b7ee-9124af131dfc None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:50.075933 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:50.085335 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940d37e5-74ab-4e6c-b7ee-9124af131dfc None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:50.092329 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b533e95e-049f-4dce-a68f-d6e5b256e000 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:50.133116 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e5e1d6-f0e8-4e66-b28c-ee7c831814a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:50.133822 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e5e1d6-f0e8-4e66-b28c-ee7c831814a4 None admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:50.134097 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e5e1d6-f0e8-4e66-b28c-ee7c831814a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:50.134208 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86ff896-480a-4bde-bf67-c74d9dce4509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:50.134384 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e5e1d6-f0e8-4e66-b28c-ee7c831814a4 None admin] PATH_INFO: `/v3/projects/c02e49a89cdd40d885bfa816f06c58a1/users/ea70acd744994250a2905b5dd7163dfb/roles/466ed51553a34a62a2c926a8bc552f68` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:50.134950 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86ff896-480a-4bde-bf67-c74d9dce4509 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:50.135406 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86ff896-480a-4bde-bf67-c74d9dce4509 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:50.135760 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86ff896-480a-4bde-bf67-c74d9dce4509 None admin] PATH_INFO: `/v3/services` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:50.136616 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e5e1d6-f0e8-4e66-b28c-ee7c831814a4 None admin] RBAC: Authorizing `identity:create_grant(project_id=c02e49a89cdd40d885bfa816f06c58a1, user_id=ea70acd744994250a2905b5dd7163dfb, role_id=466ed51553a34a62a2c926a8bc552f68)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:50.136734 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86ff896-480a-4bde-bf67-c74d9dce4509 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:50.136921 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86ff896-480a-4bde-bf67-c74d9dce4509 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:50.138142 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e5e1d6-f0e8-4e66-b28c-ee7c831814a4 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:50.138316 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86ff896-480a-4bde-bf67-c74d9dce4509 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:50.144251 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 486/959] 10.222.0.47 () {56 vars in 1122 bytes} [Wed Aug 24 06:22:49 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:50.145782 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a875e9f-911d-4975-970e-35e90c5e9af8 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:50.146069 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a875e9f-911d-4975-970e-35e90c5e9af8 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:50.146354 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a875e9f-911d-4975-970e-35e90c5e9af8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:50.166417 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-9a875e9f-911d-4975-970e-35e90c5e9af8 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:22:50.167604 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-9a875e9f-911d-4975-970e-35e90c5e9af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:22:50.191418 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:50.240190 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 474/960] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Aug 24 06:22:49 2022] PUT /identity/v3/projects/c02e49a89cdd40d885bfa816f06c58a1/users/ea70acd744994250a2905b5dd7163dfb/roles/466ed51553a34a62a2c926a8bc552f68 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:22:50.241475 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e5e1d6-f0e8-4e66-b28c-ee7c831814a4 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:50.241906 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 487/961] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:22:50 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:22:50.360699 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e5e1d6-f0e8-4e66-b28c-ee7c831814a4 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:50.400597 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aa57bb-eb6e-445f-8a26-f193b52a0fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:50.401255 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa57bb-eb6e-445f-8a26-f193b52a0fd7 None admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:50.401583 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa57bb-eb6e-445f-8a26-f193b52a0fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:50.401919 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa57bb-eb6e-445f-8a26-f193b52a0fd7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:50.402643 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aa57bb-eb6e-445f-8a26-f193b52a0fd7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:50.404044 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aa57bb-eb6e-445f-8a26-f193b52a0fd7 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:50.486479 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 475/962] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:22:50 2022] POST /identity/v3/endpoints => generated 349 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:22:50.533202 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a875e9f-911d-4975-970e-35e90c5e9af8 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:50.664721 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a875e9f-911d-4975-970e-35e90c5e9af8 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:50.711641 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac526598-c8b1-47e7-a94e-75c7dce30062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:50.712255 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac526598-c8b1-47e7-a94e-75c7dce30062 None admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:50.712534 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac526598-c8b1-47e7-a94e-75c7dce30062 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:50.712805 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac526598-c8b1-47e7-a94e-75c7dce30062 None admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:50.714538 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac526598-c8b1-47e7-a94e-75c7dce30062 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:50.715078 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac526598-c8b1-47e7-a94e-75c7dce30062 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:50.717674 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: WARNING keystone.server.flask.common [None req-ac526598-c8b1-47e7-a94e-75c7dce30062 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 24 06:22:50.771560 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.sql.core [None req-ac526598-c8b1-47e7-a94e-75c7dce30062 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 06:22:50.771560 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 24 06:22:50.771560 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [parameters: {'user_id': 'dbe0c4707af94e05a7768af1ee39cf19', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 24 06:22:50.771560 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81723) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 06:22:50.790653 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-ac526598-c8b1-47e7-a94e-75c7dce30062 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 24 06:22:50.790653 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:50.790653 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 06:22:50.790653 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 06:22:50.790653 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 06:22:50.790653 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 06:22:50.790653 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 06:22:50.790653 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 06:22:50.790653 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 06:22:50.790653 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application conn.query(q) Aug 24 06:22:50.790653 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 06:22:50.790653 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 06:22:50.790653 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 06:22:50.790653 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application result.read() Aug 24 06:22:50.790653 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 06:22:50.791639 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 06:22:50.791639 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 06:22:50.791639 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 06:22:50.791639 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 06:22:50.791639 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 06:22:50.791639 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 06:22:50.791639 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 06:22:50.791639 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 06:22:50.791639 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:50.791639 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 06:22:50.791639 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:50.791639 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:50.791639 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 06:22:50.791639 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 06:22:50.791639 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 24 06:22:50.791639 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 24 06:22:50.791639 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 06:22:50.791639 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application next(self.gen) Aug 24 06:22:50.792614 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 06:22:50.792614 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application yield resource Aug 24 06:22:50.792614 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 06:22:50.792614 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application next(self.gen) Aug 24 06:22:50.792614 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 06:22:50.792614 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 06:22:50.792614 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 06:22:50.792614 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 06:22:50.792614 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 06:22:50.792614 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise self.value Aug 24 06:22:50.792614 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 06:22:50.792614 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 06:22:50.792614 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 06:22:50.792614 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application session.commit() Aug 24 06:22:50.792614 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 06:22:50.792614 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 06:22:50.792614 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 06:22:50.802091 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 06:22:50.802091 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 06:22:50.802091 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application self.session.flush() Aug 24 06:22:50.802091 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 06:22:50.802091 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 06:22:50.802091 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 06:22:50.802091 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 06:22:50.802091 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 06:22:50.802091 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application compat.raise_( Aug 24 06:22:50.802091 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 06:22:50.802091 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception Aug 24 06:22:50.802091 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 06:22:50.802091 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 06:22:50.802091 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 06:22:50.802091 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 06:22:50.802091 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 06:22:50.802091 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 06:22:50.804277 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 06:22:50.804277 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 06:22:50.804277 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 24 06:22:50.804277 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 24 06:22:50.804277 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 06:22:50.804277 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 06:22:50.804277 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 06:22:50.804277 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 06:22:50.804277 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 06:22:50.804277 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 06:22:50.804277 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 06:22:50.804277 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 06:22:50.804277 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 06:22:50.804277 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 06:22:50.804277 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 06:22:50.804277 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception Aug 24 06:22:50.806371 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 06:22:50.806371 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 06:22:50.806371 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 06:22:50.806371 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 06:22:50.806371 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 06:22:50.806371 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 06:22:50.806371 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 06:22:50.806371 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application conn.query(q) Aug 24 06:22:50.806371 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 06:22:50.806371 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 06:22:50.806371 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 06:22:50.806371 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application result.read() Aug 24 06:22:50.806371 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 06:22:50.806371 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 06:22:50.806371 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 06:22:50.806371 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 06:22:50.808377 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 06:22:50.808377 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 06:22:50.808377 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 06:22:50.808377 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 06:22:50.808377 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 06:22:50.808377 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 24 06:22:50.808377 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [parameters: {'user_id': 'dbe0c4707af94e05a7768af1ee39cf19', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 24 06:22:50.808377 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 06:22:50.808377 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:50.808377 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 06:22:50.808377 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:50.808377 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:50.808377 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:50.808377 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:50.808377 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:50.814507 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:50.814507 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:50.814507 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:50.814507 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:50.814507 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:50.814507 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:50.814507 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:50.814507 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 24 06:22:50.814507 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 24 06:22:50.814507 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:50.814507 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:50.814507 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 06:22:50.814507 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:22:50.814507 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 06:22:50.814507 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:22:50.814507 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 24 06:22:50.814507 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 24 06:22:50.816767 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 24 06:22:50.816767 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 24 06:22:50.816767 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 06:22:50.816767 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 06:22:50.816767 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 24 06:22:50.816767 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:50.816767 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 488/963] 10.222.0.47 () {60 vars in 1131 bytes} [Wed Aug 24 06:22:50 2022] POST /identity/v3/users => generated 160 bytes in 260 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:50.822839 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aa57bb-eb6e-445f-8a26-f193b52a0fd7 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:50.841572 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:50.935917 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aa57bb-eb6e-445f-8a26-f193b52a0fd7 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:50.973206 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553e91f5-d3e6-46d7-94b1-b6fba7e611a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:50.973870 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553e91f5-d3e6-46d7-94b1-b6fba7e611a1 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:50.974154 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553e91f5-d3e6-46d7-94b1-b6fba7e611a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:50.974435 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553e91f5-d3e6-46d7-94b1-b6fba7e611a1 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:50.982675 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553e91f5-d3e6-46d7-94b1-b6fba7e611a1 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:50.984144 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553e91f5-d3e6-46d7-94b1-b6fba7e611a1 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:50.997436 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-553e91f5-d3e6-46d7-94b1-b6fba7e611a1 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 06:22:50.997436 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:50.997436 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:50.997436 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:50.997436 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:50.997436 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:50.997436 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:50.997436 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:50.997436 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:50.997436 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:50.997436 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:50.997436 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:50.997436 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 06:22:50.997436 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 06:22:50.997436 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 06:22:50.997436 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 06:22:50.999757 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:50.999757 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:50.999757 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 06:22:50.999757 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:22:50.999757 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 06:22:50.999757 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:22:50.999757 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:50.999757 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:50.999757 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:50.999757 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:50.999757 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:50.999757 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:50.999757 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:50.999757 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:50.999757 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:50.999757 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:50.999757 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:50.999757 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:51.002000 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:51.002000 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:51.002000 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 06:22:51.002000 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 06:22:51.002000 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 06:22:51.002000 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 06:22:51.002000 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 06:22:51.002000 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 06:22:51.002000 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 06:22:51.002000 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:51.002000 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 476/964] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 24 06:22:50 2022] GET /identity/v3/users/cinder => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:51.010926 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac526598-c8b1-47e7-a94e-75c7dce30062 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:51.038871 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:51.124110 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac526598-c8b1-47e7-a94e-75c7dce30062 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:51.162633 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c584df-e9bf-41a9-8b89-e72ce4e242df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:51.163875 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c584df-e9bf-41a9-8b89-e72ce4e242df None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:51.164168 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c584df-e9bf-41a9-8b89-e72ce4e242df None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:51.164426 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c584df-e9bf-41a9-8b89-e72ce4e242df None admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:51.165134 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c584df-e9bf-41a9-8b89-e72ce4e242df None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:51.165465 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c584df-e9bf-41a9-8b89-e72ce4e242df None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:51.167172 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c584df-e9bf-41a9-8b89-e72ce4e242df None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:51.208271 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 489/965] 10.222.0.47 () {56 vars in 1101 bytes} [Wed Aug 24 06:22:51 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:52.029165 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a8f4c8-27a1-456e-bc00-1e4c23bb37c6 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:52.029678 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a8f4c8-27a1-456e-bc00-1e4c23bb37c6 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:52.029678 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a8f4c8-27a1-456e-bc00-1e4c23bb37c6 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:52.031003 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 477/966] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:22:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:22:52.038395 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f143aa-615e-4b9e-954c-5ce42acc43ae None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:52.039120 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f143aa-615e-4b9e-954c-5ce42acc43ae None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:52.039120 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f143aa-615e-4b9e-954c-5ce42acc43ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:52.059617 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-77f143aa-615e-4b9e-954c-5ce42acc43ae None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:22:52.061127 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-77f143aa-615e-4b9e-954c-5ce42acc43ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:22:52.133236 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 490/967] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:22:52 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:22:52.269277 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8287027e-9f7d-4ca0-8241-c65226673ece None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:52.269609 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8287027e-9f7d-4ca0-8241-c65226673ece None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:52.269788 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8287027e-9f7d-4ca0-8241-c65226673ece None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:52.271490 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 478/968] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:22:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:22:52.277461 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f585d794-5285-432f-b47e-b284fe02a36d None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:52.277757 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f585d794-5285-432f-b47e-b284fe02a36d None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:52.278039 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f585d794-5285-432f-b47e-b284fe02a36d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:52.285276 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:52.296333 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-f585d794-5285-432f-b47e-b284fe02a36d None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:22:52.297607 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-f585d794-5285-432f-b47e-b284fe02a36d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:22:52.369218 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 491/969] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:22:52 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:22:52.405457 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8287027e-9f7d-4ca0-8241-c65226673ece None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:52.477448 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:52.510434 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8287027e-9f7d-4ca0-8241-c65226673ece None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:52.539241 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d55a28e-cee4-4f65-be7a-32c1d806f7c3 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:52.539784 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d55a28e-cee4-4f65-be7a-32c1d806f7c3 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:52.540066 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d55a28e-cee4-4f65-be7a-32c1d806f7c3 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:52.541561 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 492/970] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:22:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 06:22:52.547209 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d35a68-4898-45b5-8f3d-947ff9d0c97b None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:52.547501 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d35a68-4898-45b5-8f3d-947ff9d0c97b None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:52.547786 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d35a68-4898-45b5-8f3d-947ff9d0c97b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:52.549010 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc8a938-dc29-4d27-bfda-20362e8afc87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:52.549936 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc8a938-dc29-4d27-bfda-20362e8afc87 None admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:52.550419 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc8a938-dc29-4d27-bfda-20362e8afc87 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:52.550759 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc8a938-dc29-4d27-bfda-20362e8afc87 None admin] PATH_INFO: `/v3/services` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:52.551600 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc8a938-dc29-4d27-bfda-20362e8afc87 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:52.553171 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc8a938-dc29-4d27-bfda-20362e8afc87 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:52.566813 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-94d35a68-4898-45b5-8f3d-947ff9d0c97b None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:22:52.568055 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-94d35a68-4898-45b5-8f3d-947ff9d0c97b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:22:52.635715 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 479/971] 10.222.0.47 () {60 vars in 1141 bytes} [Wed Aug 24 06:22:52 2022] POST /identity/v3/services => generated 242 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:22:52.644152 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc8a938-dc29-4d27-bfda-20362e8afc87 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:52.653239 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 493/972] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:22:52 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:22:52.767858 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc8a938-dc29-4d27-bfda-20362e8afc87 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:52.801948 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:52.808387 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4017e4c8-0922-4999-b5fd-5a2e866545d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:52.809043 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4017e4c8-0922-4999-b5fd-5a2e866545d7 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:52.809374 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4017e4c8-0922-4999-b5fd-5a2e866545d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:52.809651 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4017e4c8-0922-4999-b5fd-5a2e866545d7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:52.822498 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4017e4c8-0922-4999-b5fd-5a2e866545d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:22:52.827215 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-4017e4c8-0922-4999-b5fd-5a2e866545d7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 06:22:52.827215 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:52.827215 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:52.827215 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:52.827215 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:52.827215 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:52.827215 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:52.827215 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:52.827215 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:52.827215 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:52.827215 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:52.827215 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:52.827215 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:22:52.827215 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:22:52.827215 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:22:52.827215 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:22:52.828178 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:22:52.828178 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:22:52.828178 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:22:52.828178 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:22:52.828178 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:52.828178 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:52.828178 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:52.828178 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:52.828178 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:52.828178 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:52.828178 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:52.828178 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:52.828178 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:52.828178 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:52.828178 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:52.828178 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:52.829065 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:52.829065 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:52.829065 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:52.829065 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:52.829065 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:22:52.829065 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:22:52.829065 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:22:52.829065 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:22:52.829065 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:22:52.829065 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:22:52.829065 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 06:22:52.829065 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:52.829065 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 480/973] 10.222.0.47 () {56 vars in 1120 bytes} [Wed Aug 24 06:22:52 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:22:52.834360 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d35a68-4898-45b5-8f3d-947ff9d0c97b None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:52.899050 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:52.933843 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4017e4c8-0922-4999-b5fd-5a2e866545d7 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:52.940549 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d35a68-4898-45b5-8f3d-947ff9d0c97b None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:52.980382 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce85f004-48a0-470f-b6cf-682050ae1e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:52.981002 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce85f004-48a0-470f-b6cf-682050ae1e3b None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:52.981309 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce85f004-48a0-470f-b6cf-682050ae1e3b None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:52.981696 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce85f004-48a0-470f-b6cf-682050ae1e3b None admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:52.982485 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce85f004-48a0-470f-b6cf-682050ae1e3b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:52.982792 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce85f004-48a0-470f-b6cf-682050ae1e3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:52.984161 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce85f004-48a0-470f-b6cf-682050ae1e3b None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:52.998002 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 494/974] 10.222.0.47 () {56 vars in 1115 bytes} [Wed Aug 24 06:22:52 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:52.999090 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce85f004-48a0-470f-b6cf-682050ae1e3b None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:53.042675 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4017e4c8-0922-4999-b5fd-5a2e866545d7 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:53.079112 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a756ca3-4e59-4831-b397-809918e0db57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:53.079680 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a756ca3-4e59-4831-b397-809918e0db57 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:53.079941 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a756ca3-4e59-4831-b397-809918e0db57 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:53.080213 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a756ca3-4e59-4831-b397-809918e0db57 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:53.080793 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a756ca3-4e59-4831-b397-809918e0db57 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:53.082211 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a756ca3-4e59-4831-b397-809918e0db57 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:53.091675 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-5a756ca3-4e59-4831-b397-809918e0db57 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 06:22:53.091675 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:53.091675 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 06:22:53.091675 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 06:22:53.091675 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 06:22:53.091675 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 06:22:53.091675 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 06:22:53.091675 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:22:53.091675 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 06:22:53.091675 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:53.091675 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 06:22:53.091675 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:53.091675 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:53.091675 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:53.091675 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:53.091675 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:53.091675 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:53.092764 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:53.092764 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:53.092764 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:53.092764 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:53.092764 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:53.092764 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:53.092764 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 06:22:53.092764 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 06:22:53.092764 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 06:22:53.092764 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 06:22:53.092764 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:53.092764 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:53.092764 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:53.092764 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:53.092764 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:53.092764 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:53.093670 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:53.093670 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:53.093670 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:53.093670 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:53.093670 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:53.093670 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:53.093670 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:53.093670 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:53.093670 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:53.093670 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:53.093670 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 06:22:53.093670 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:22:53.093670 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 06:22:53.093670 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:53.093670 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 481/975] 10.222.0.47 () {56 vars in 1126 bytes} [Wed Aug 24 06:22:52 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:22:53.098647 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a756ca3-4e59-4831-b397-809918e0db57 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:53.115863 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce85f004-48a0-470f-b6cf-682050ae1e3b None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:53.142095 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:53.155102 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595b86c6-fbda-4516-931a-e8fc1dd2dad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:53.159829 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595b86c6-fbda-4516-931a-e8fc1dd2dad2 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:53.159829 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595b86c6-fbda-4516-931a-e8fc1dd2dad2 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:53.159829 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595b86c6-fbda-4516-931a-e8fc1dd2dad2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:53.165033 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:53.168570 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595b86c6-fbda-4516-931a-e8fc1dd2dad2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:53.170233 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595b86c6-fbda-4516-931a-e8fc1dd2dad2 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:53.186911 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-595b86c6-fbda-4516-931a-e8fc1dd2dad2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:53.186911 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:53.186911 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:53.186911 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:53.186911 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:53.186911 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:53.186911 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:53.186911 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:53.186911 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:53.186911 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:53.186911 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:53.186911 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:53.186911 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:22:53.186911 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:22:53.186911 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:22:53.186911 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:22:53.187937 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:53.187937 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:53.187937 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:53.187937 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:53.187937 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:53.187937 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:53.187937 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:53.187937 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:53.187937 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:53.187937 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:53.187937 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:53.187937 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:53.187937 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:53.187937 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:53.187937 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:53.187937 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:53.187937 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:22:53.189861 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:22:53.189861 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:53.189861 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:53.189861 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 495/976] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:22:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:53.194912 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595b86c6-fbda-4516-931a-e8fc1dd2dad2 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:53.213431 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a756ca3-4e59-4831-b397-809918e0db57 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:53.248729 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5894cb-85b6-4246-afa8-391be5ea8233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:53.249375 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5894cb-85b6-4246-afa8-391be5ea8233 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:53.249650 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5894cb-85b6-4246-afa8-391be5ea8233 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:53.249913 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5894cb-85b6-4246-afa8-391be5ea8233 None admin] PATH_INFO: `/v3/services` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:53.250546 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5894cb-85b6-4246-afa8-391be5ea8233 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:53.250852 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5894cb-85b6-4246-afa8-391be5ea8233 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:53.252204 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5894cb-85b6-4246-afa8-391be5ea8233 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:53.264120 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 482/977] 10.222.0.47 () {56 vars in 1122 bytes} [Wed Aug 24 06:22:53 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:22:53.265028 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5894cb-85b6-4246-afa8-391be5ea8233 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:53.320971 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595b86c6-fbda-4516-931a-e8fc1dd2dad2 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:53.357716 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e478588-3f73-4315-8665-ac907c21e778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:53.358337 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e478588-3f73-4315-8665-ac907c21e778 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:53.358622 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e478588-3f73-4315-8665-ac907c21e778 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:53.358898 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e478588-3f73-4315-8665-ac907c21e778 None admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:53.359559 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e478588-3f73-4315-8665-ac907c21e778 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:53.359863 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e478588-3f73-4315-8665-ac907c21e778 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:53.361258 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e478588-3f73-4315-8665-ac907c21e778 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:53.379230 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 496/978] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 24 06:22:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:22:53.380389 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e478588-3f73-4315-8665-ac907c21e778 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:53.388876 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5894cb-85b6-4246-afa8-391be5ea8233 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:53.409761 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:53.417269 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:53.429549 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb0a90d-df71-4c10-b119-f3a56d402beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:53.430169 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb0a90d-df71-4c10-b119-f3a56d402beb None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:53.430446 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb0a90d-df71-4c10-b119-f3a56d402beb None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:53.430721 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb0a90d-df71-4c10-b119-f3a56d402beb None admin] PATH_INFO: `/v3/services` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:53.431347 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb0a90d-df71-4c10-b119-f3a56d402beb None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:53.431644 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb0a90d-df71-4c10-b119-f3a56d402beb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:53.433534 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb0a90d-df71-4c10-b119-f3a56d402beb None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:53.439288 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 483/979] 10.222.0.47 () {56 vars in 1122 bytes} [Wed Aug 24 06:22:53 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:53.447033 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb0a90d-df71-4c10-b119-f3a56d402beb None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:53.499098 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e478588-3f73-4315-8665-ac907c21e778 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:53.545677 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f181d8-e0d4-4b20-88c4-2b68a96d9034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:53.546320 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f181d8-e0d4-4b20-88c4-2b68a96d9034 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:53.546604 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f181d8-e0d4-4b20-88c4-2b68a96d9034 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:53.546877 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f181d8-e0d4-4b20-88c4-2b68a96d9034 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:53.554990 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f181d8-e0d4-4b20-88c4-2b68a96d9034 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:53.556451 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f181d8-e0d4-4b20-88c4-2b68a96d9034 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:53.565653 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb0a90d-df71-4c10-b119-f3a56d402beb None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:53.568509 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-78f181d8-e0d4-4b20-88c4-2b68a96d9034 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 06:22:53.568509 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:53.568509 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:53.568509 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:53.568509 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:53.568509 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:53.568509 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:53.568509 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:53.568509 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:53.568509 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:53.568509 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:53.568509 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:53.568509 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 06:22:53.568509 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 06:22:53.568509 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 06:22:53.568509 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 06:22:53.569752 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:53.569752 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:53.569752 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 06:22:53.569752 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:22:53.569752 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 06:22:53.569752 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:22:53.569752 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:53.569752 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:53.569752 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:53.569752 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:53.569752 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:53.569752 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:53.569752 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:53.569752 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:53.569752 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:53.569752 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:53.569752 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:53.569752 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:53.571336 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:53.571336 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:53.571336 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 06:22:53.571336 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 06:22:53.571336 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 06:22:53.571336 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 06:22:53.571336 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 06:22:53.571336 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 06:22:53.571336 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 06:22:53.571336 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:53.571336 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 497/980] 10.222.0.47 () {56 vars in 1117 bytes} [Wed Aug 24 06:22:53 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:53.575877 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f181d8-e0d4-4b20-88c4-2b68a96d9034 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:53.614787 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f661f316-56b7-435d-acac-86917bc1bcbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:53.615411 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f661f316-56b7-435d-acac-86917bc1bcbe None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:53.615751 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f661f316-56b7-435d-acac-86917bc1bcbe None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:53.616118 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f661f316-56b7-435d-acac-86917bc1bcbe None admin] PATH_INFO: `/v3/endpoints` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:53.617038 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f661f316-56b7-435d-acac-86917bc1bcbe None admin] RBAC: Adding query filter params (interface=admin, service_id=7399165cb4f346abb275646eb6bd45e8, region_id=RegionOne) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:53.617389 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f661f316-56b7-435d-acac-86917bc1bcbe None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:53.618765 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f661f316-56b7-435d-acac-86917bc1bcbe None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:53.627483 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 484/981] 10.222.0.47 () {56 vars in 1249 bytes} [Wed Aug 24 06:22:53 2022] GET /identity/v3/endpoints?service_id=7399165cb4f346abb275646eb6bd45e8&interface=admin®ion_id=RegionOne => generated 193 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:22:53.633461 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f661f316-56b7-435d-acac-86917bc1bcbe None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:53.694638 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:53.698075 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:53.703116 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f181d8-e0d4-4b20-88c4-2b68a96d9034 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:53.739125 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b3cb0a-afba-4dfa-8752-eb6b3a045cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:53.739747 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b3cb0a-afba-4dfa-8752-eb6b3a045cea None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:53.740019 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b3cb0a-afba-4dfa-8752-eb6b3a045cea None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:53.740292 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b3cb0a-afba-4dfa-8752-eb6b3a045cea None admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:53.741553 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b3cb0a-afba-4dfa-8752-eb6b3a045cea None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:53.742349 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b3cb0a-afba-4dfa-8752-eb6b3a045cea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:53.744112 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b3cb0a-afba-4dfa-8752-eb6b3a045cea None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:53.755703 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f661f316-56b7-435d-acac-86917bc1bcbe None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:53.763169 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 498/982] 10.222.0.47 () {56 vars in 1149 bytes} [Wed Aug 24 06:22:53 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:22:53.769139 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b3cb0a-afba-4dfa-8752-eb6b3a045cea None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:53.799162 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec19ff7-b884-4ff6-84bd-c72d80f3d736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:53.799768 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec19ff7-b884-4ff6-84bd-c72d80f3d736 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:53.800077 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec19ff7-b884-4ff6-84bd-c72d80f3d736 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:53.800323 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec19ff7-b884-4ff6-84bd-c72d80f3d736 None admin] PATH_INFO: `/v3/services` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:53.800945 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec19ff7-b884-4ff6-84bd-c72d80f3d736 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:53.802317 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec19ff7-b884-4ff6-84bd-c72d80f3d736 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:53.811983 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 485/983] 10.222.0.47 () {56 vars in 1087 bytes} [Wed Aug 24 06:22:53 2022] GET /identity/v3/services => generated 1727 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:22:53.887012 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b3cb0a-afba-4dfa-8752-eb6b3a045cea None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:53.925762 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95340100-745b-4f97-bfbf-ce9caa731e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:53.926397 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95340100-745b-4f97-bfbf-ce9caa731e67 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:53.926680 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95340100-745b-4f97-bfbf-ce9caa731e67 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:53.926955 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95340100-745b-4f97-bfbf-ce9caa731e67 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:53.952623 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:54.009600 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95340100-745b-4f97-bfbf-ce9caa731e67 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:54.011310 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95340100-745b-4f97-bfbf-ce9caa731e67 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:54.126195 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 499/984] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 24 06:22:53 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 06:22:54.132818 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec19ff7-b884-4ff6-84bd-c72d80f3d736 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:54.235416 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec19ff7-b884-4ff6-84bd-c72d80f3d736 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:54.246687 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:54.270977 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6560a7-a32a-4cf3-9b1e-f469207e5ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:54.271618 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6560a7-a32a-4cf3-9b1e-f469207e5ee4 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:54.272028 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6560a7-a32a-4cf3-9b1e-f469207e5ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:54.272394 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6560a7-a32a-4cf3-9b1e-f469207e5ee4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:54.284008 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6560a7-a32a-4cf3-9b1e-f469207e5ee4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:54.285509 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6560a7-a32a-4cf3-9b1e-f469207e5ee4 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:54.299996 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-6f6560a7-a32a-4cf3-9b1e-f469207e5ee4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:54.299996 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:54.299996 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:54.299996 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:54.299996 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:54.299996 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:54.299996 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:54.299996 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:54.299996 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:54.299996 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:54.299996 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:54.299996 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:54.299996 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:22:54.299996 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:22:54.299996 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:22:54.299996 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:22:54.300990 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:54.300990 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:54.300990 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:54.300990 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:54.300990 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:54.300990 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:54.300990 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:54.300990 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:54.300990 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:54.300990 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:54.300990 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:54.300990 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:54.300990 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:54.300990 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:54.300990 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:54.300990 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:54.300990 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:22:54.302026 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:22:54.302026 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:22:54.302026 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:54.305538 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 486/985] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:22:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:54.307469 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95340100-745b-4f97-bfbf-ce9caa731e67 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:54.361579 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:54.411395 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95340100-745b-4f97-bfbf-ce9caa731e67 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:54.447197 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04d4135-b8cc-4697-b36e-c60454385f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:54.447849 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04d4135-b8cc-4697-b36e-c60454385f31 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:54.448129 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04d4135-b8cc-4697-b36e-c60454385f31 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:54.448402 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04d4135-b8cc-4697-b36e-c60454385f31 None admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:54.449045 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04d4135-b8cc-4697-b36e-c60454385f31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:54.449453 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04d4135-b8cc-4697-b36e-c60454385f31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:54.450828 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04d4135-b8cc-4697-b36e-c60454385f31 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:54.467325 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 500/986] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 24 06:22:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:22:54.468993 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6560a7-a32a-4cf3-9b1e-f469207e5ee4 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:54.570785 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6560a7-a32a-4cf3-9b1e-f469207e5ee4 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:54.604683 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92de546-160a-4a73-814b-f3f41ede7db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:54.605350 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92de546-160a-4a73-814b-f3f41ede7db5 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:54.605634 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92de546-160a-4a73-814b-f3f41ede7db5 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:54.605908 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92de546-160a-4a73-814b-f3f41ede7db5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:54.612488 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92de546-160a-4a73-814b-f3f41ede7db5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:54.613950 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92de546-160a-4a73-814b-f3f41ede7db5 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:54.626885 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-f92de546-160a-4a73-814b-f3f41ede7db5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:22:54.626885 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:54.626885 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:54.626885 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:54.626885 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:54.626885 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:54.626885 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:54.626885 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:54.626885 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:54.626885 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:54.626885 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:54.626885 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:54.626885 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 06:22:54.626885 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 06:22:54.626885 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 06:22:54.627816 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 06:22:54.627816 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:54.627816 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:54.627816 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:54.627816 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:54.627816 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:54.627816 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:54.627816 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:54.627816 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:54.627816 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:54.627816 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:54.627816 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:54.627816 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:54.627816 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:54.627816 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:54.627816 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:54.627816 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:54.628701 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 06:22:54.628701 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 06:22:54.628701 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:22:54.628701 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:22:54.628701 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:22:54.628701 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:22:54.628701 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:22:54.628701 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:22:54.629319 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbebd3e-1cbe-4702-9696-7f06a16def64 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:54.629319 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbebd3e-1cbe-4702-9696-7f06a16def64 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:54.629319 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbebd3e-1cbe-4702-9696-7f06a16def64 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:54.633213 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 501/987] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:22:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:22:54.633339 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 487/988] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 24 06:22:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:22:54.634241 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbebd3e-1cbe-4702-9696-7f06a16def64 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:54.635384 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b72b192-8ef9-42ca-a301-5200e95ef23e None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:54.635677 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b72b192-8ef9-42ca-a301-5200e95ef23e None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:54.635967 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b72b192-8ef9-42ca-a301-5200e95ef23e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:54.655691 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-6b72b192-8ef9-42ca-a301-5200e95ef23e None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:22:54.657249 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-6b72b192-8ef9-42ca-a301-5200e95ef23e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:22:54.673838 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:54.739274 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 488/989] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:22:54 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 06:22:54.755868 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbebd3e-1cbe-4702-9696-7f06a16def64 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:54.770432 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:54.792564 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fc86bc-44da-49d8-9fb1-5af504f7945e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:54.793199 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fc86bc-44da-49d8-9fb1-5af504f7945e None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:54.793496 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fc86bc-44da-49d8-9fb1-5af504f7945e None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:54.793823 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fc86bc-44da-49d8-9fb1-5af504f7945e None admin] PATH_INFO: `/v3/projects` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:54.794522 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fc86bc-44da-49d8-9fb1-5af504f7945e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:54.794831 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fc86bc-44da-49d8-9fb1-5af504f7945e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:54.796272 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fc86bc-44da-49d8-9fb1-5af504f7945e None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:54.809997 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 502/990] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Aug 24 06:22:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:54.815835 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b72b192-8ef9-42ca-a301-5200e95ef23e None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:54.923050 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b72b192-8ef9-42ca-a301-5200e95ef23e None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:54.959723 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708b55c4-12f1-4b4e-b301-450795b8d2c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:54.960407 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708b55c4-12f1-4b4e-b301-450795b8d2c3 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:54.960689 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708b55c4-12f1-4b4e-b301-450795b8d2c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:54.961024 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708b55c4-12f1-4b4e-b301-450795b8d2c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:54.961804 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708b55c4-12f1-4b4e-b301-450795b8d2c3 None admin] RBAC: Adding query filter params (role.id=466ed51553a34a62a2c926a8bc552f68, scope.project.id=c02e49a89cdd40d885bfa816f06c58a1, user.id=ea70acd744994250a2905b5dd7163dfb) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:54.962115 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708b55c4-12f1-4b4e-b301-450795b8d2c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:54.963542 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708b55c4-12f1-4b4e-b301-450795b8d2c3 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:54.977227 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 489/991] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 24 06:22:54 2022] GET /identity/v3/role_assignments?user.id=ea70acd744994250a2905b5dd7163dfb&scope.project.id=c02e49a89cdd40d885bfa816f06c58a1&role.id=466ed51553a34a62a2c926a8bc552f68 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:55.009672 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fc86bc-44da-49d8-9fb1-5af504f7945e None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:55.121921 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fc86bc-44da-49d8-9fb1-5af504f7945e None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:55.158586 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3cba4b-ff5c-4044-82c3-fc726277f310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:55.159229 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3cba4b-ff5c-4044-82c3-fc726277f310 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:55.159482 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3cba4b-ff5c-4044-82c3-fc726277f310 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:55.159758 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3cba4b-ff5c-4044-82c3-fc726277f310 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:55.160339 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3cba4b-ff5c-4044-82c3-fc726277f310 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:55.161764 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3cba4b-ff5c-4044-82c3-fc726277f310 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:55.170636 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-8b3cba4b-ff5c-4044-82c3-fc726277f310 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 06:22:55.170636 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:55.170636 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 06:22:55.170636 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 06:22:55.170636 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 06:22:55.170636 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 06:22:55.170636 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 06:22:55.170636 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:22:55.170636 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 06:22:55.170636 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:55.170636 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 06:22:55.170636 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:55.170636 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:55.170636 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:55.170636 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:55.170636 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:55.170636 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:55.171941 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:55.171941 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:55.171941 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:55.171941 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:55.171941 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:55.171941 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:55.171941 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 06:22:55.171941 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 06:22:55.171941 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 06:22:55.171941 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 06:22:55.171941 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:55.171941 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:55.171941 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:55.171941 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:55.171941 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:55.171941 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:55.172934 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:55.172934 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:55.172934 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:55.172934 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:55.172934 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:55.172934 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:55.172934 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:55.172934 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:55.172934 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:55.172934 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:55.172934 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 06:22:55.172934 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:22:55.172934 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 06:22:55.172934 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:55.176797 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 503/992] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 24 06:22:55 2022] GET /identity/v3/services/compute => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:55.178665 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708b55c4-12f1-4b4e-b301-450795b8d2c3 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:55.198417 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:55.293986 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708b55c4-12f1-4b4e-b301-450795b8d2c3 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:55.329342 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16db86dc-abc2-4305-ad1a-9988a7b1fea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:55.329959 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16db86dc-abc2-4305-ad1a-9988a7b1fea7 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:55.330242 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16db86dc-abc2-4305-ad1a-9988a7b1fea7 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:55.330516 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16db86dc-abc2-4305-ad1a-9988a7b1fea7 None admin] PATH_INFO: `/v3/services` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:55.331143 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16db86dc-abc2-4305-ad1a-9988a7b1fea7 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:55.331446 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16db86dc-abc2-4305-ad1a-9988a7b1fea7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:55.332807 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16db86dc-abc2-4305-ad1a-9988a7b1fea7 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:55.342441 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 490/993] 10.222.0.47 () {56 vars in 1112 bytes} [Wed Aug 24 06:22:55 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:55.343590 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3cba4b-ff5c-4044-82c3-fc726277f310 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:55.385400 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:55.467065 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3cba4b-ff5c-4044-82c3-fc726277f310 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:55.507945 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc7e672-e6f2-4fca-8ac5-245bc1464a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:55.508558 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc7e672-e6f2-4fca-8ac5-245bc1464a45 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:55.508840 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc7e672-e6f2-4fca-8ac5-245bc1464a45 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:55.509115 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc7e672-e6f2-4fca-8ac5-245bc1464a45 None admin] PATH_INFO: `/v3/services` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:55.509778 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc7e672-e6f2-4fca-8ac5-245bc1464a45 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:55.510083 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc7e672-e6f2-4fca-8ac5-245bc1464a45 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:55.511462 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc7e672-e6f2-4fca-8ac5-245bc1464a45 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:55.522558 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 504/994] 10.222.0.47 () {56 vars in 1112 bytes} [Wed Aug 24 06:22:55 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:55.523532 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16db86dc-abc2-4305-ad1a-9988a7b1fea7 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:55.613350 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:55.637538 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16db86dc-abc2-4305-ad1a-9988a7b1fea7 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:55.683497 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa389ab-3749-4080-8821-913565947051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:55.684116 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa389ab-3749-4080-8821-913565947051 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:55.684399 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa389ab-3749-4080-8821-913565947051 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:55.684676 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa389ab-3749-4080-8821-913565947051 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:55.685405 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa389ab-3749-4080-8821-913565947051 None admin] RBAC: Adding query filter params (interface=public, service_id=6db67e45471e447c8fd969c13dd81354, region_id=RegionOne) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:55.685711 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa389ab-3749-4080-8821-913565947051 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:55.687053 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa389ab-3749-4080-8821-913565947051 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:55.697158 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 491/995] 10.222.0.47 () {56 vars in 1251 bytes} [Wed Aug 24 06:22:55 2022] GET /identity/v3/endpoints?service_id=6db67e45471e447c8fd969c13dd81354&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:55.698784 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc7e672-e6f2-4fca-8ac5-245bc1464a45 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:55.793639 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1952dfa-e8e1-4163-9100-51db515fe365 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:55.793940 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1952dfa-e8e1-4163-9100-51db515fe365 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:55.794120 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1952dfa-e8e1-4163-9100-51db515fe365 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:55.795533 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 492/996] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:22:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 06:22:55.800557 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cd1fe1-d919-4a09-9700-5773ddb80ffd None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:55.800842 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cd1fe1-d919-4a09-9700-5773ddb80ffd None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:55.801117 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cd1fe1-d919-4a09-9700-5773ddb80ffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:55.818274 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:55.821793 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc7e672-e6f2-4fca-8ac5-245bc1464a45 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:55.823063 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-48cd1fe1-d919-4a09-9700-5773ddb80ffd None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:22:55.824297 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-48cd1fe1-d919-4a09-9700-5773ddb80ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:22:55.860842 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3304ddb7-a95c-4805-97f6-2b1e4e10867d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:55.861476 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3304ddb7-a95c-4805-97f6-2b1e4e10867d None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:55.861755 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3304ddb7-a95c-4805-97f6-2b1e4e10867d None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:55.862037 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3304ddb7-a95c-4805-97f6-2b1e4e10867d None admin] PATH_INFO: `/v3/services` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:55.862642 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3304ddb7-a95c-4805-97f6-2b1e4e10867d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:55.864001 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3304ddb7-a95c-4805-97f6-2b1e4e10867d None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:55.877244 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 505/997] 10.222.0.47 () {56 vars in 1087 bytes} [Wed Aug 24 06:22:55 2022] GET /identity/v3/services => generated 1727 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:22:55.913563 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 493/998] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:22:55 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 06:22:56.183488 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3304ddb7-a95c-4805-97f6-2b1e4e10867d None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:56.290735 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3304ddb7-a95c-4805-97f6-2b1e4e10867d None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:56.326327 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b95c8a7-5a8c-4ee6-810c-397e76420aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:56.326327 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b95c8a7-5a8c-4ee6-810c-397e76420aa9 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:56.326649 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b95c8a7-5a8c-4ee6-810c-397e76420aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:56.326938 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b95c8a7-5a8c-4ee6-810c-397e76420aa9 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:56.327410 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b95c8a7-5a8c-4ee6-810c-397e76420aa9 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:56.328774 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b95c8a7-5a8c-4ee6-810c-397e76420aa9 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:56.338767 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-2b95c8a7-5a8c-4ee6-810c-397e76420aa9 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 06:22:56.338767 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:56.338767 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 06:22:56.338767 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 06:22:56.338767 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 06:22:56.338767 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 06:22:56.338767 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 06:22:56.338767 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:22:56.338767 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 06:22:56.338767 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:56.338767 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 06:22:56.338767 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:56.338767 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:56.338767 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:56.338767 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:56.338767 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:56.338767 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:56.339943 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:56.339943 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:56.339943 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:56.339943 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:56.339943 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:56.339943 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:56.339943 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 06:22:56.339943 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 06:22:56.339943 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 06:22:56.339943 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 06:22:56.339943 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:56.339943 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:56.339943 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:56.339943 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:56.339943 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:56.339943 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:56.341659 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:56.341659 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:56.341659 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:56.341659 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:56.341659 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:56.341659 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:56.341659 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:56.341659 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:56.341659 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:56.341659 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:56.341659 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 06:22:56.341659 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:22:56.341659 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 06:22:56.341659 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:56.341659 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 506/999] 10.222.0.47 () {56 vars in 1126 bytes} [Wed Aug 24 06:22:56 2022] GET /identity/v3/services/object-store => generated 93 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:56.346768 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cd1fe1-d919-4a09-9700-5773ddb80ffd None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:56.430033 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:56.455572 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cd1fe1-d919-4a09-9700-5773ddb80ffd None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:56.493037 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238c28e7-ca9d-4911-912b-17ad1c443936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:56.493695 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c28e7-ca9d-4911-912b-17ad1c443936 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:56.493982 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c28e7-ca9d-4911-912b-17ad1c443936 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:56.494257 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c28e7-ca9d-4911-912b-17ad1c443936 None admin] PATH_INFO: `/v3/services` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:56.494900 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238c28e7-ca9d-4911-912b-17ad1c443936 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:56.495206 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238c28e7-ca9d-4911-912b-17ad1c443936 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:56.496587 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238c28e7-ca9d-4911-912b-17ad1c443936 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:56.502057 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 494/1000] 10.222.0.47 () {56 vars in 1122 bytes} [Wed Aug 24 06:22:56 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:56.507609 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b95c8a7-5a8c-4ee6-810c-397e76420aa9 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:56.539834 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:56.614748 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b95c8a7-5a8c-4ee6-810c-397e76420aa9 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:56.660078 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf7c203-254e-4653-a7bf-cb4497827add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:56.660756 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf7c203-254e-4653-a7bf-cb4497827add None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:56.660970 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf7c203-254e-4653-a7bf-cb4497827add None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:56.661274 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf7c203-254e-4653-a7bf-cb4497827add None admin] PATH_INFO: `/v3/services` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:56.662040 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf7c203-254e-4653-a7bf-cb4497827add None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:56.662928 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf7c203-254e-4653-a7bf-cb4497827add None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:56.664407 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf7c203-254e-4653-a7bf-cb4497827add None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:56.674517 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 507/1001] 10.222.0.47 () {56 vars in 1122 bytes} [Wed Aug 24 06:22:56 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:56.675866 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238c28e7-ca9d-4911-912b-17ad1c443936 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:56.796933 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238c28e7-ca9d-4911-912b-17ad1c443936 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:56.825193 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64983a89-9708-4a4c-96d3-ede5f7fc213a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:56.825852 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64983a89-9708-4a4c-96d3-ede5f7fc213a None admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:56.826134 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64983a89-9708-4a4c-96d3-ede5f7fc213a None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:56.826411 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64983a89-9708-4a4c-96d3-ede5f7fc213a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:56.827064 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64983a89-9708-4a4c-96d3-ede5f7fc213a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:56.828412 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64983a89-9708-4a4c-96d3-ede5f7fc213a None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:56.943062 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:56.950098 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 495/1002] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:22:56 2022] POST /identity/v3/endpoints => generated 325 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:22:57.012950 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5d64ec-6963-4760-9c9c-e1df8b7cc539 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:57.013310 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5d64ec-6963-4760-9c9c-e1df8b7cc539 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:57.013540 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5d64ec-6963-4760-9c9c-e1df8b7cc539 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:57.015249 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 508/1003] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:22:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:22:57.021287 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab175c55-c5a9-40fa-9381-cb2cab01da6c None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:57.021710 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab175c55-c5a9-40fa-9381-cb2cab01da6c None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:57.022094 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab175c55-c5a9-40fa-9381-cb2cab01da6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:57.042592 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-ab175c55-c5a9-40fa-9381-cb2cab01da6c None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:22:57.043816 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-ab175c55-c5a9-40fa-9381-cb2cab01da6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:22:57.126105 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 496/1004] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:22:57 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:22:57.399478 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5d64ec-6963-4760-9c9c-e1df8b7cc539 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:57.498520 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:57.509590 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5d64ec-6963-4760-9c9c-e1df8b7cc539 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:57.545847 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce576c6f-e1db-466a-86a4-2bde1a47053e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:57.546479 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce576c6f-e1db-466a-86a4-2bde1a47053e None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:57.546759 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce576c6f-e1db-466a-86a4-2bde1a47053e None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:57.547033 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce576c6f-e1db-466a-86a4-2bde1a47053e None admin] PATH_INFO: `/v3/services/image` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:57.547610 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce576c6f-e1db-466a-86a4-2bde1a47053e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:57.548976 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce576c6f-e1db-466a-86a4-2bde1a47053e None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:57.558640 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-ce576c6f-e1db-466a-86a4-2bde1a47053e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 06:22:57.558640 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:57.558640 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 06:22:57.558640 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 06:22:57.558640 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 06:22:57.558640 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 06:22:57.558640 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 06:22:57.558640 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:22:57.558640 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 06:22:57.558640 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:57.558640 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 06:22:57.558640 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:57.558640 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:57.558640 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:57.558640 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:57.558640 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:57.558640 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:57.559993 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:57.559993 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:57.559993 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:57.559993 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:57.559993 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:57.559993 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:57.559993 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 06:22:57.559993 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 06:22:57.559993 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 06:22:57.559993 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 06:22:57.559993 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:57.559993 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:57.559993 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:57.559993 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:57.559993 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:57.559993 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:57.560865 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:57.560865 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:57.560865 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:57.560865 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:57.560865 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:57.560865 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:57.560865 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:57.560865 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:57.560865 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:57.560865 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:57.560865 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 06:22:57.560865 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:22:57.560865 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 06:22:57.560865 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:57.564708 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 509/1005] 10.222.0.47 () {56 vars in 1105 bytes} [Wed Aug 24 06:22:57 2022] GET /identity/v3/services/image => generated 86 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:57.565992 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab175c55-c5a9-40fa-9381-cb2cab01da6c None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:57.675010 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab175c55-c5a9-40fa-9381-cb2cab01da6c None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:57.699618 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:57.711084 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6815972a-ea75-483b-aebb-490bd3a5990f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:57.711704 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6815972a-ea75-483b-aebb-490bd3a5990f None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:57.711987 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6815972a-ea75-483b-aebb-490bd3a5990f None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:57.712263 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6815972a-ea75-483b-aebb-490bd3a5990f None admin] PATH_INFO: `/v3/services` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:57.718604 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6815972a-ea75-483b-aebb-490bd3a5990f None admin] RBAC: Adding query filter params (name=image) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:57.718604 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6815972a-ea75-483b-aebb-490bd3a5990f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:57.718604 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6815972a-ea75-483b-aebb-490bd3a5990f None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:57.723853 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 497/1006] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:22:57 2022] GET /identity/v3/services?name=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:57.725227 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce576c6f-e1db-466a-86a4-2bde1a47053e None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:57.831540 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce576c6f-e1db-466a-86a4-2bde1a47053e None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:57.849701 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd72e11-7011-4bd0-ba78-9821a09cdf49 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:57.849985 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd72e11-7011-4bd0-ba78-9821a09cdf49 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:57.850263 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd72e11-7011-4bd0-ba78-9821a09cdf49 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:57.851749 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 498/1007] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:22:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 06:22:57.856847 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5fd8a3-c8e3-4fd5-80d7-e33bae96f4f3 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:57.857134 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5fd8a3-c8e3-4fd5-80d7-e33bae96f4f3 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:57.857439 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5fd8a3-c8e3-4fd5-80d7-e33bae96f4f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:57.869011 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208eb9ad-ba04-425d-aad7-4aa6ccf2d11a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:57.869640 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208eb9ad-ba04-425d-aad7-4aa6ccf2d11a None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:57.869918 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208eb9ad-ba04-425d-aad7-4aa6ccf2d11a None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:57.870191 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208eb9ad-ba04-425d-aad7-4aa6ccf2d11a None admin] PATH_INFO: `/v3/services` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:57.870820 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208eb9ad-ba04-425d-aad7-4aa6ccf2d11a None admin] RBAC: Adding query filter params (type=image) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:57.871121 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208eb9ad-ba04-425d-aad7-4aa6ccf2d11a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:57.872476 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208eb9ad-ba04-425d-aad7-4aa6ccf2d11a None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:57.876342 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-1d5fd8a3-c8e3-4fd5-80d7-e33bae96f4f3 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:22:57.877016 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 510/1008] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:22:57 2022] GET /identity/v3/services?type=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:57.878046 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-1d5fd8a3-c8e3-4fd5-80d7-e33bae96f4f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:22:57.963737 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 499/1009] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:22:57 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 06:22:58.248085 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208eb9ad-ba04-425d-aad7-4aa6ccf2d11a None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:58.273035 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:58.358474 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208eb9ad-ba04-425d-aad7-4aa6ccf2d11a None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:58.396777 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57697c73-f4ce-4552-b9b5-89526a369e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:58.397466 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57697c73-f4ce-4552-b9b5-89526a369e59 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:58.397751 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57697c73-f4ce-4552-b9b5-89526a369e59 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:58.398024 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57697c73-f4ce-4552-b9b5-89526a369e59 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:58.398621 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57697c73-f4ce-4552-b9b5-89526a369e59 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:58.400024 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57697c73-f4ce-4552-b9b5-89526a369e59 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:58.409689 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-57697c73-f4ce-4552-b9b5-89526a369e59 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 06:22:58.409689 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:58.409689 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 06:22:58.409689 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 06:22:58.409689 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 06:22:58.409689 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 06:22:58.409689 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 06:22:58.409689 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:22:58.409689 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 06:22:58.409689 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:58.409689 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 06:22:58.409689 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:58.409689 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:22:58.409689 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:22:58.409689 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:22:58.409689 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:22:58.409689 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:22:58.410739 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:22:58.410739 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:22:58.410739 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:22:58.410739 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:22:58.410739 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:22:58.410739 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:22:58.410739 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 06:22:58.410739 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 06:22:58.410739 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 06:22:58.410739 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 06:22:58.410739 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:22:58.410739 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:22:58.410739 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:22:58.410739 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:22:58.410739 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:22:58.410739 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:22:58.411589 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:22:58.411589 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:22:58.411589 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:22:58.411589 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:22:58.411589 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:22:58.411589 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:22:58.411589 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:22:58.411589 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:22:58.411589 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:22:58.411589 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:22:58.411589 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 06:22:58.411589 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:22:58.411589 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 06:22:58.411589 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:22:58.411589 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 511/1010] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 24 06:22:58 2022] GET /identity/v3/services/compute => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:22:58.418192 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5fd8a3-c8e3-4fd5-80d7-e33bae96f4f3 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:58.527217 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5fd8a3-c8e3-4fd5-80d7-e33bae96f4f3 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:58.546068 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:58.564041 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843c6123-b01e-4f78-9f9d-42f7f18d275a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:58.564665 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843c6123-b01e-4f78-9f9d-42f7f18d275a None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:58.564949 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843c6123-b01e-4f78-9f9d-42f7f18d275a None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:58.565245 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843c6123-b01e-4f78-9f9d-42f7f18d275a None admin] PATH_INFO: `/v3/services` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:58.565883 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843c6123-b01e-4f78-9f9d-42f7f18d275a None admin] RBAC: Adding query filter params (name=compute) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:58.566186 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843c6123-b01e-4f78-9f9d-42f7f18d275a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:58.567546 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843c6123-b01e-4f78-9f9d-42f7f18d275a None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:58.577273 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 500/1011] 10.222.0.47 () {56 vars in 1112 bytes} [Wed Aug 24 06:22:58 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:58.578920 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57697c73-f4ce-4552-b9b5-89526a369e59 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:58.675558 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:58.692049 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57697c73-f4ce-4552-b9b5-89526a369e59 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:58.728110 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852662a1-5024-4c8c-b007-b933e40d40b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:58.728778 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852662a1-5024-4c8c-b007-b933e40d40b5 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:58.729065 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852662a1-5024-4c8c-b007-b933e40d40b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:58.729402 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852662a1-5024-4c8c-b007-b933e40d40b5 None admin] PATH_INFO: `/v3/services` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:58.730046 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852662a1-5024-4c8c-b007-b933e40d40b5 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:22:58.730350 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852662a1-5024-4c8c-b007-b933e40d40b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:58.731728 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852662a1-5024-4c8c-b007-b933e40d40b5 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:58.741305 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 512/1012] 10.222.0.47 () {56 vars in 1112 bytes} [Wed Aug 24 06:22:58 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:22:58.744558 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843c6123-b01e-4f78-9f9d-42f7f18d275a None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:58.897286 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843c6123-b01e-4f78-9f9d-42f7f18d275a None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:58.955339 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00ceb2e-b7be-46ef-9797-20dc3caa3e99 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:58.955848 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00ceb2e-b7be-46ef-9797-20dc3caa3e99 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:58.956354 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00ceb2e-b7be-46ef-9797-20dc3caa3e99 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:58.957296 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb009279-9866-42c7-b5d8-29b7a42eb333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:58.957962 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb009279-9866-42c7-b5d8-29b7a42eb333 None admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:58.958200 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb009279-9866-42c7-b5d8-29b7a42eb333 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:58.958471 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb009279-9866-42c7-b5d8-29b7a42eb333 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:58.961222 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 513/1013] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:22:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:22:58.961374 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb009279-9866-42c7-b5d8-29b7a42eb333 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:58.961374 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb009279-9866-42c7-b5d8-29b7a42eb333 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:58.966400 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6924a0cf-eb4f-4947-ab87-776c626e6b63 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:58.966974 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6924a0cf-eb4f-4947-ab87-776c626e6b63 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:58.967330 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6924a0cf-eb4f-4947-ab87-776c626e6b63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:58.988602 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-6924a0cf-eb4f-4947-ab87-776c626e6b63 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:22:58.989872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-6924a0cf-eb4f-4947-ab87-776c626e6b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:22:59.068184 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 514/1014] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:22:58 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:22:59.075481 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 501/1015] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:22:58 2022] POST /identity/v3/endpoints => generated 334 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:22:59.353565 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6924a0cf-eb4f-4947-ab87-776c626e6b63 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:59.442086 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:59.466101 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6924a0cf-eb4f-4947-ab87-776c626e6b63 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:59.509956 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48365c7f-5c7c-4844-8671-e0d5733d3631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:59.510597 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48365c7f-5c7c-4844-8671-e0d5733d3631 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:59.510975 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48365c7f-5c7c-4844-8671-e0d5733d3631 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:59.511171 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48365c7f-5c7c-4844-8671-e0d5733d3631 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:59.512309 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48365c7f-5c7c-4844-8671-e0d5733d3631 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:59.513782 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48365c7f-5c7c-4844-8671-e0d5733d3631 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:59.515599 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 515/1016] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:22:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:22:59.522118 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb009279-9866-42c7-b5d8-29b7a42eb333 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:22:59.566889 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:22:59.634292 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb009279-9866-42c7-b5d8-29b7a42eb333 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:22:59.671876 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a1addb-3491-4663-a033-e81b546fc3b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:22:59.672498 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a1addb-3491-4663-a033-e81b546fc3b0 None admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:59.672779 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a1addb-3491-4663-a033-e81b546fc3b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:59.673068 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a1addb-3491-4663-a033-e81b546fc3b0 None admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:59.673847 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a1addb-3491-4663-a033-e81b546fc3b0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:22:59.675284 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a1addb-3491-4663-a033-e81b546fc3b0 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:22:59.773219 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 502/1017] 10.222.0.47 () {60 vars in 1141 bytes} [Wed Aug 24 06:22:59 2022] POST /identity/v3/projects => generated 311 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:22:59.842384 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d25d3a-198f-47f5-bb6f-a387b6658873 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:59.842618 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d25d3a-198f-47f5-bb6f-a387b6658873 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:59.842896 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d25d3a-198f-47f5-bb6f-a387b6658873 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:59.844526 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 516/1018] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:22:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:22:59.850708 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c10dd15-75cc-48b6-bd7e-e0fd488c7a18 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:22:59.850991 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c10dd15-75cc-48b6-bd7e-e0fd488c7a18 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:22:59.851271 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c10dd15-75cc-48b6-bd7e-e0fd488c7a18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:22:59.873314 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-1c10dd15-75cc-48b6-bd7e-e0fd488c7a18 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:22:59.874456 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-1c10dd15-75cc-48b6-bd7e-e0fd488c7a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:22:59.954507 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 503/1019] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:22:59 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:23:00.224360 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d25d3a-198f-47f5-bb6f-a387b6658873 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:00.342163 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d25d3a-198f-47f5-bb6f-a387b6658873 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:00.383276 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4fe110-6c95-41df-919b-d1b10f4aa5f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:00.383896 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4fe110-6c95-41df-919b-d1b10f4aa5f5 None admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:00.384275 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4fe110-6c95-41df-919b-d1b10f4aa5f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:00.384594 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4fe110-6c95-41df-919b-d1b10f4aa5f5 None admin] PATH_INFO: `/v3/services` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:00.385287 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4fe110-6c95-41df-919b-d1b10f4aa5f5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:00.387152 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4fe110-6c95-41df-919b-d1b10f4aa5f5 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:00.509297 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 517/1020] 10.222.0.47 () {60 vars in 1141 bytes} [Wed Aug 24 06:23:00 2022] POST /identity/v3/services => generated 242 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:23:01.192969 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fe01a2-a3c6-4971-9ce0-08dc985bf253 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:01.194136 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fe01a2-a3c6-4971-9ce0-08dc985bf253 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:01.194136 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fe01a2-a3c6-4971-9ce0-08dc985bf253 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:01.195588 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 504/1021] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:23:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:23:01.201367 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9b2931-af2b-4c9f-bf8a-fe50b411e5aa None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:01.201652 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9b2931-af2b-4c9f-bf8a-fe50b411e5aa None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:01.201923 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9b2931-af2b-4c9f-bf8a-fe50b411e5aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:01.208132 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:01.221797 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-1b9b2931-af2b-4c9f-bf8a-fe50b411e5aa None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:23:01.233047 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-1b9b2931-af2b-4c9f-bf8a-fe50b411e5aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:23:01.304128 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 518/1022] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:23:01 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:23:01.584059 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fe01a2-a3c6-4971-9ce0-08dc985bf253 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:01.623234 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:01.728324 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fe01a2-a3c6-4971-9ce0-08dc985bf253 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:01.775954 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8062cab2-780e-42c7-9a5b-eb36ab340a9e None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:01.776469 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8062cab2-780e-42c7-9a5b-eb36ab340a9e None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:01.776646 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8062cab2-780e-42c7-9a5b-eb36ab340a9e None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:01.777469 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00afaecc-5120-4131-9412-a5348614d341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:01.778178 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 519/1023] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:23:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:23:01.778302 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00afaecc-5120-4131-9412-a5348614d341 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:01.778379 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00afaecc-5120-4131-9412-a5348614d341 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:01.778635 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00afaecc-5120-4131-9412-a5348614d341 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:01.784164 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454c27b8-1b17-4ba2-af80-e037c9d1f4fe None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:01.784688 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454c27b8-1b17-4ba2-af80-e037c9d1f4fe None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:01.785165 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454c27b8-1b17-4ba2-af80-e037c9d1f4fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:01.796825 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00afaecc-5120-4131-9412-a5348614d341 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:23:01.801842 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-00afaecc-5120-4131-9412-a5348614d341 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 06:23:01.801842 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:23:01.801842 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:23:01.801842 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:23:01.801842 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:23:01.801842 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:23:01.801842 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:23:01.801842 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:23:01.801842 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:23:01.801842 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:23:01.801842 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:23:01.801842 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:23:01.801842 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:23:01.801842 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:23:01.801842 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:23:01.801842 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:23:01.803575 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:23:01.803575 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:23:01.803575 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:23:01.803575 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:23:01.803575 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:23:01.803575 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:23:01.803575 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:23:01.803575 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:23:01.803575 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:23:01.803575 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:23:01.803575 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:23:01.803575 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:23:01.803575 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:23:01.803575 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:23:01.803575 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:23:01.803575 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:23:01.804405 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:23:01.804405 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:23:01.804405 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:23:01.804405 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:23:01.804405 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:23:01.804405 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:23:01.804405 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:23:01.804405 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:23:01.804405 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:23:01.804405 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:23:01.804405 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 06:23:01.804405 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:23:01.804405 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 505/1024] 10.222.0.47 () {56 vars in 1120 bytes} [Wed Aug 24 06:23:01 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:23:01.808177 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-454c27b8-1b17-4ba2-af80-e037c9d1f4fe None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:23:01.809613 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-454c27b8-1b17-4ba2-af80-e037c9d1f4fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:23:01.810682 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00afaecc-5120-4131-9412-a5348614d341 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:01.891941 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 520/1025] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:23:01 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 06:23:01.916632 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:01.951353 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00afaecc-5120-4131-9412-a5348614d341 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:02.010706 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4ac81e-b783-4864-8fef-c7234a88d65f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:02.011726 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4ac81e-b783-4864-8fef-c7234a88d65f None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:02.012212 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4ac81e-b783-4864-8fef-c7234a88d65f None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:02.012700 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4ac81e-b783-4864-8fef-c7234a88d65f None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:02.013928 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4ac81e-b783-4864-8fef-c7234a88d65f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:23:02.014451 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4ac81e-b783-4864-8fef-c7234a88d65f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:02.017019 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4ac81e-b783-4864-8fef-c7234a88d65f None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:02.038738 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 506/1026] 10.222.0.47 () {56 vars in 1115 bytes} [Wed Aug 24 06:23:01 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:23:02.040443 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454c27b8-1b17-4ba2-af80-e037c9d1f4fe None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:02.108197 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:02.155924 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454c27b8-1b17-4ba2-af80-e037c9d1f4fe None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:02.166138 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4ac81e-b783-4864-8fef-c7234a88d65f None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:02.202158 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dceef17-d543-49aa-93a9-b503c642c0ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:02.205659 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dceef17-d543-49aa-93a9-b503c642c0ca None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:02.206389 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dceef17-d543-49aa-93a9-b503c642c0ca None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:02.207211 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dceef17-d543-49aa-93a9-b503c642c0ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:02.218531 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dceef17-d543-49aa-93a9-b503c642c0ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:02.220019 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dceef17-d543-49aa-93a9-b503c642c0ca None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:02.235757 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-1dceef17-d543-49aa-93a9-b503c642c0ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:23:02.235757 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:23:02.235757 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:23:02.235757 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:23:02.235757 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:23:02.235757 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:23:02.235757 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:23:02.235757 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:23:02.235757 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:23:02.235757 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:23:02.235757 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:23:02.235757 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:23:02.235757 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:23:02.235757 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:23:02.235757 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:23:02.235757 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:23:02.236963 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:23:02.236963 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:23:02.236963 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:23:02.236963 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:23:02.236963 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:23:02.236963 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:23:02.236963 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:23:02.236963 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:23:02.236963 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:23:02.236963 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:23:02.236963 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:23:02.236963 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:23:02.236963 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:23:02.236963 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:23:02.236963 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:23:02.236963 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:23:02.236963 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:23:02.238059 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:23:02.238059 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:23:02.238059 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:23:02.238059 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 521/1027] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:23:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:23:02.243466 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dceef17-d543-49aa-93a9-b503c642c0ca None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:02.299067 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4ac81e-b783-4864-8fef-c7234a88d65f None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:02.347582 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dc8959-ecc3-4109-b4ca-54eb4a6f81af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:02.348498 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc8959-ecc3-4109-b4ca-54eb4a6f81af None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:02.348942 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc8959-ecc3-4109-b4ca-54eb4a6f81af None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:02.349835 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dc8959-ecc3-4109-b4ca-54eb4a6f81af None admin] PATH_INFO: `/v3/domains/default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:02.351050 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dc8959-ecc3-4109-b4ca-54eb4a6f81af None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:02.352458 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dc8959-ecc3-4109-b4ca-54eb4a6f81af None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:02.357619 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 507/1028] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:23:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:23:02.359741 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dc8959-ecc3-4109-b4ca-54eb4a6f81af None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:02.372369 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dceef17-d543-49aa-93a9-b503c642c0ca None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:02.377306 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:02.390965 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:02.422639 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec46081f-4c42-4fd1-a7e8-ed0594e79561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:02.423556 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec46081f-4c42-4fd1-a7e8-ed0594e79561 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:02.424027 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec46081f-4c42-4fd1-a7e8-ed0594e79561 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:02.424305 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec46081f-4c42-4fd1-a7e8-ed0594e79561 None admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:02.425001 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec46081f-4c42-4fd1-a7e8-ed0594e79561 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:23:02.425467 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec46081f-4c42-4fd1-a7e8-ed0594e79561 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:02.426831 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec46081f-4c42-4fd1-a7e8-ed0594e79561 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:02.446270 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 522/1029] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 24 06:23:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:23:02.448157 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec46081f-4c42-4fd1-a7e8-ed0594e79561 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:02.489826 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dc8959-ecc3-4109-b4ca-54eb4a6f81af None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:02.548127 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e00101-3cfc-428f-9d3a-82e8b1d44984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:02.549073 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e00101-3cfc-428f-9d3a-82e8b1d44984 None admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:02.549557 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e00101-3cfc-428f-9d3a-82e8b1d44984 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:02.549994 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e00101-3cfc-428f-9d3a-82e8b1d44984 None admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:02.551061 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e00101-3cfc-428f-9d3a-82e8b1d44984 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:02.553927 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e00101-3cfc-428f-9d3a-82e8b1d44984 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:02.587969 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec46081f-4c42-4fd1-a7e8-ed0594e79561 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:02.623007 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e3e50d-add8-4c73-a6fb-dbc437d30070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:02.623639 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e3e50d-add8-4c73-a6fb-dbc437d30070 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:02.623916 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e3e50d-add8-4c73-a6fb-dbc437d30070 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:02.624185 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e3e50d-add8-4c73-a6fb-dbc437d30070 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:02.632083 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e3e50d-add8-4c73-a6fb-dbc437d30070 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:02.633548 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e3e50d-add8-4c73-a6fb-dbc437d30070 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:02.645470 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-a2e3e50d-add8-4c73-a6fb-dbc437d30070 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 06:23:02.645470 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:23:02.645470 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:23:02.645470 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:23:02.645470 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:23:02.645470 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:23:02.645470 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:23:02.645470 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:23:02.645470 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:23:02.645470 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:23:02.645470 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:23:02.645470 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:23:02.645470 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 06:23:02.645470 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 06:23:02.645470 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 06:23:02.645470 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 06:23:02.647147 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:23:02.647147 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:23:02.647147 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 06:23:02.647147 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:23:02.647147 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 06:23:02.647147 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:23:02.647147 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:23:02.647147 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:23:02.647147 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:23:02.647147 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:23:02.647147 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:23:02.647147 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:23:02.647147 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:23:02.647147 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:23:02.647147 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:23:02.647147 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:23:02.647147 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:23:02.647147 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:23:02.648477 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:23:02.648477 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:23:02.648477 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 06:23:02.648477 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 06:23:02.648477 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 06:23:02.648477 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 06:23:02.648477 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 06:23:02.648477 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 06:23:02.648477 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 06:23:02.648477 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:23:02.648477 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 523/1030] 10.222.0.47 () {56 vars in 1093 bytes} [Wed Aug 24 06:23:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:23:02.648477 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19051a4-b221-48f1-ae84-86c70a3a381c None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:02.649278 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19051a4-b221-48f1-ae84-86c70a3a381c None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:02.649278 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19051a4-b221-48f1-ae84-86c70a3a381c None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:02.650441 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 524/1031] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:23:02 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:23:02.653005 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19051a4-b221-48f1-ae84-86c70a3a381c None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:02.685358 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:02.690507 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 508/1032] 10.222.0.47 () {60 vars in 1132 bytes} [Wed Aug 24 06:23:02 2022] POST /identity/v3/users => generated 285 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:23:02.691751 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84056e84-5b39-47e6-954c-d362bec0b1b2 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:02.693705 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84056e84-5b39-47e6-954c-d362bec0b1b2 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:02.693705 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84056e84-5b39-47e6-954c-d362bec0b1b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:02.725668 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-84056e84-5b39-47e6-954c-d362bec0b1b2 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:23:02.725668 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-84056e84-5b39-47e6-954c-d362bec0b1b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:23:02.777773 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19051a4-b221-48f1-ae84-86c70a3a381c None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:02.795333 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:02.803570 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 509/1033] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:23:02 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:23:02.820373 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d80f29-9d21-45f1-b781-2d2810870f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:02.821004 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d80f29-9d21-45f1-b781-2d2810870f8f None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:02.821336 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d80f29-9d21-45f1-b781-2d2810870f8f None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:02.821610 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d80f29-9d21-45f1-b781-2d2810870f8f None admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:02.822362 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d80f29-9d21-45f1-b781-2d2810870f8f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:23:02.822672 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d80f29-9d21-45f1-b781-2d2810870f8f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:02.824149 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d80f29-9d21-45f1-b781-2d2810870f8f None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:02.851275 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 525/1034] 10.222.0.47 () {56 vars in 1133 bytes} [Wed Aug 24 06:23:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:23:02.858018 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84056e84-5b39-47e6-954c-d362bec0b1b2 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:02.969274 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84056e84-5b39-47e6-954c-d362bec0b1b2 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:03.004808 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eada0195-ed5d-4d41-9147-aa179cb03d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:03.005978 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada0195-ed5d-4d41-9147-aa179cb03d3c None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:03.006625 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada0195-ed5d-4d41-9147-aa179cb03d3c None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:03.006625 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada0195-ed5d-4d41-9147-aa179cb03d3c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:03.073739 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:03.074192 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d80f29-9d21-45f1-b781-2d2810870f8f None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:03.082821 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eada0195-ed5d-4d41-9147-aa179cb03d3c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:03.084652 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eada0195-ed5d-4d41-9147-aa179cb03d3c None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:03.194898 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:03.215230 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d80f29-9d21-45f1-b781-2d2810870f8f None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:03.241562 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 510/1035] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 24 06:23:02 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 06:23:03.242513 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eada0195-ed5d-4d41-9147-aa179cb03d3c None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:03.258798 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d4933c-2e79-44e5-a4f8-3fba029bfe10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:03.259443 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d4933c-2e79-44e5-a4f8-3fba029bfe10 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:03.259723 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d4933c-2e79-44e5-a4f8-3fba029bfe10 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:03.259995 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d4933c-2e79-44e5-a4f8-3fba029bfe10 None admin] PATH_INFO: `/v3/services/image` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:03.260585 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d4933c-2e79-44e5-a4f8-3fba029bfe10 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:03.261987 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d4933c-2e79-44e5-a4f8-3fba029bfe10 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:03.272766 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-45d4933c-2e79-44e5-a4f8-3fba029bfe10 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 06:23:03.272766 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:23:03.272766 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 06:23:03.272766 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 06:23:03.272766 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 06:23:03.272766 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 06:23:03.272766 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 06:23:03.272766 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:23:03.272766 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 06:23:03.272766 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:23:03.272766 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 06:23:03.272766 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:23:03.272766 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:23:03.272766 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:23:03.272766 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:23:03.272766 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:23:03.272766 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:23:03.274080 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:23:03.274080 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:23:03.274080 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:23:03.274080 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:23:03.274080 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:23:03.274080 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:23:03.274080 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 06:23:03.274080 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 06:23:03.274080 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 06:23:03.274080 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 06:23:03.274080 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:23:03.274080 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:23:03.274080 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:23:03.274080 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:23:03.274080 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:23:03.274080 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:23:03.274951 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:23:03.274951 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:23:03.274951 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:23:03.274951 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:23:03.274951 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:23:03.274951 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:23:03.274951 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:23:03.274951 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:23:03.274951 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:23:03.274951 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:23:03.274951 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 06:23:03.274951 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:23:03.274951 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 06:23:03.274951 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:23:03.274951 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 526/1036] 10.222.0.47 () {56 vars in 1105 bytes} [Wed Aug 24 06:23:03 2022] GET /identity/v3/services/image => generated 86 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:23:03.280085 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d4933c-2e79-44e5-a4f8-3fba029bfe10 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:03.353122 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:03.358770 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eada0195-ed5d-4d41-9147-aa179cb03d3c None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:03.397890 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f11171-c247-4c3c-a1f2-fc5df990469b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:03.398501 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d4933c-2e79-44e5-a4f8-3fba029bfe10 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:03.398617 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f11171-c247-4c3c-a1f2-fc5df990469b None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:03.398792 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f11171-c247-4c3c-a1f2-fc5df990469b None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:03.399056 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f11171-c247-4c3c-a1f2-fc5df990469b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:03.409946 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f11171-c247-4c3c-a1f2-fc5df990469b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:03.411963 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f11171-c247-4c3c-a1f2-fc5df990469b None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:03.428267 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-84f11171-c247-4c3c-a1f2-fc5df990469b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:23:03.428267 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:23:03.428267 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:23:03.428267 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:23:03.428267 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:23:03.428267 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:23:03.428267 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:23:03.428267 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:23:03.428267 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:23:03.428267 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:23:03.428267 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:23:03.428267 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:23:03.428267 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:23:03.428267 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:23:03.428267 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:23:03.428267 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:23:03.429333 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:23:03.429333 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:23:03.429333 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:23:03.429333 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:23:03.429333 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:23:03.429333 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:23:03.429333 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:23:03.429333 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:23:03.429333 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:23:03.429333 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:23:03.429333 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:23:03.429333 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:23:03.429333 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:23:03.429333 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:23:03.429333 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:23:03.429333 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:23:03.429333 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:23:03.430250 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:23:03.430250 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:23:03.430250 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:23:03.430250 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 511/1037] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:23:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:23:03.435159 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f11171-c247-4c3c-a1f2-fc5df990469b None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:03.439365 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec88125-6882-41c2-9061-ad13c664eaf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:03.439982 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec88125-6882-41c2-9061-ad13c664eaf1 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:03.440262 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec88125-6882-41c2-9061-ad13c664eaf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:03.440535 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec88125-6882-41c2-9061-ad13c664eaf1 None admin] PATH_INFO: `/v3/services` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:03.441167 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec88125-6882-41c2-9061-ad13c664eaf1 None admin] RBAC: Adding query filter params (name=image) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:23:03.441568 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec88125-6882-41c2-9061-ad13c664eaf1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:03.443568 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec88125-6882-41c2-9061-ad13c664eaf1 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:03.453588 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 527/1038] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:23:03 2022] GET /identity/v3/services?name=image => generated 122 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:23:03.454447 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec88125-6882-41c2-9061-ad13c664eaf1 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:03.473955 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:03.547882 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f11171-c247-4c3c-a1f2-fc5df990469b None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:03.564333 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec88125-6882-41c2-9061-ad13c664eaf1 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:03.587680 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50815824-b2e4-425c-b26b-b90b6f466f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:03.588301 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50815824-b2e4-425c-b26b-b90b6f466f0d None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:03.588578 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50815824-b2e4-425c-b26b-b90b6f466f0d None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:03.588848 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50815824-b2e4-425c-b26b-b90b6f466f0d None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:03.589508 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50815824-b2e4-425c-b26b-b90b6f466f0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:23:03.589816 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50815824-b2e4-425c-b26b-b90b6f466f0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:03.591173 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50815824-b2e4-425c-b26b-b90b6f466f0d None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:03.603742 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 512/1039] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 24 06:23:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:23:03.609213 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50815824-b2e4-425c-b26b-b90b6f466f0d None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:03.609663 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0773ba0-3756-4851-a7bd-7e1b1124f5fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:03.610222 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0773ba0-3756-4851-a7bd-7e1b1124f5fc None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:03.610497 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0773ba0-3756-4851-a7bd-7e1b1124f5fc None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:03.610770 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0773ba0-3756-4851-a7bd-7e1b1124f5fc None admin] PATH_INFO: `/v3/services` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:03.611400 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0773ba0-3756-4851-a7bd-7e1b1124f5fc None admin] RBAC: Adding query filter params (type=image) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:23:03.611697 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0773ba0-3756-4851-a7bd-7e1b1124f5fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:03.613055 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0773ba0-3756-4851-a7bd-7e1b1124f5fc None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:03.619446 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 528/1040] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:23:03 2022] GET /identity/v3/services?type=image => generated 350 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:23:03.625083 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0773ba0-3756-4851-a7bd-7e1b1124f5fc None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:03.627454 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:03.724531 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50815824-b2e4-425c-b26b-b90b6f466f0d None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:03.736773 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:03.768550 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5029322c-002b-47cf-8e36-aa1e1c7caf18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:03.769469 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5029322c-002b-47cf-8e36-aa1e1c7caf18 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:03.769580 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0773ba0-3756-4851-a7bd-7e1b1124f5fc None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:03.769745 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5029322c-002b-47cf-8e36-aa1e1c7caf18 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:03.770020 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5029322c-002b-47cf-8e36-aa1e1c7caf18 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:03.778922 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5029322c-002b-47cf-8e36-aa1e1c7caf18 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:03.780838 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5029322c-002b-47cf-8e36-aa1e1c7caf18 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:03.793824 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-5029322c-002b-47cf-8e36-aa1e1c7caf18 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:23:03.793824 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:23:03.793824 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:23:03.793824 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:23:03.793824 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:23:03.793824 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:23:03.793824 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:23:03.793824 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:23:03.793824 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:23:03.793824 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:23:03.793824 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:23:03.793824 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:23:03.793824 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 06:23:03.793824 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 06:23:03.793824 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 06:23:03.794775 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 06:23:03.794775 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:23:03.794775 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:23:03.794775 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:23:03.794775 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:23:03.794775 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:23:03.794775 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:23:03.794775 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:23:03.794775 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:23:03.794775 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:23:03.794775 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:23:03.794775 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:23:03.794775 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:23:03.794775 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:23:03.794775 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:23:03.794775 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:23:03.794775 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:23:03.795688 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 06:23:03.795688 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 06:23:03.795688 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:23:03.795688 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:23:03.795688 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:23:03.795688 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:23:03.795688 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:23:03.795688 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:23:03.795688 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 513/1041] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 24 06:23:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:23:03.800745 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5029322c-002b-47cf-8e36-aa1e1c7caf18 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:03.811352 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522dc791-080f-4f5c-bc9b-dc332f9d86a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:03.812011 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522dc791-080f-4f5c-bc9b-dc332f9d86a8 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:03.812249 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522dc791-080f-4f5c-bc9b-dc332f9d86a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:03.812531 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522dc791-080f-4f5c-bc9b-dc332f9d86a8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:03.813259 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522dc791-080f-4f5c-bc9b-dc332f9d86a8 None admin] RBAC: Adding query filter params (interface=public, service_id=614bcaa7cae548eb8964f161f5d88e3a, region_id=RegionOne) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:23:03.813567 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522dc791-080f-4f5c-bc9b-dc332f9d86a8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:03.814924 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522dc791-080f-4f5c-bc9b-dc332f9d86a8 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:03.826515 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 529/1042] 10.222.0.47 () {56 vars in 1251 bytes} [Wed Aug 24 06:23:03 2022] GET /identity/v3/endpoints?service_id=614bcaa7cae548eb8964f161f5d88e3a&interface=public®ion_id=RegionOne => generated 194 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:23:03.828048 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522dc791-080f-4f5c-bc9b-dc332f9d86a8 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:03.884830 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:03.923653 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5029322c-002b-47cf-8e36-aa1e1c7caf18 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:03.948231 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522dc791-080f-4f5c-bc9b-dc332f9d86a8 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:03.960450 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c48db5-8f07-4107-89b0-617ca0df7e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:03.960814 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c48db5-8f07-4107-89b0-617ca0df7e65 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:03.961092 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c48db5-8f07-4107-89b0-617ca0df7e65 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:03.961392 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c48db5-8f07-4107-89b0-617ca0df7e65 None admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:03.962081 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c48db5-8f07-4107-89b0-617ca0df7e65 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:23:03.962383 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c48db5-8f07-4107-89b0-617ca0df7e65 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:03.963799 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c48db5-8f07-4107-89b0-617ca0df7e65 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:03.976621 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 514/1043] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Aug 24 06:23:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:23:03.982522 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c48db5-8f07-4107-89b0-617ca0df7e65 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:03.988460 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d0b023-3070-402e-836c-d48c4af4f063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:03.989331 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d0b023-3070-402e-836c-d48c4af4f063 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:03.990409 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d0b023-3070-402e-836c-d48c4af4f063 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:03.990928 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d0b023-3070-402e-836c-d48c4af4f063 None admin] PATH_INFO: `/v3/services` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:03.992169 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d0b023-3070-402e-836c-d48c4af4f063 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:03.994137 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d0b023-3070-402e-836c-d48c4af4f063 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:04.003272 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 530/1044] 10.222.0.47 () {56 vars in 1087 bytes} [Wed Aug 24 06:23:03 2022] GET /identity/v3/services => generated 1957 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:23:04.094957 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c48db5-8f07-4107-89b0-617ca0df7e65 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:04.127050 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:04.136893 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29c3060-1737-47d8-b8bf-e25ae0dd20fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:04.137649 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29c3060-1737-47d8-b8bf-e25ae0dd20fc None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:04.137887 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29c3060-1737-47d8-b8bf-e25ae0dd20fc None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:04.138158 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29c3060-1737-47d8-b8bf-e25ae0dd20fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:04.138913 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29c3060-1737-47d8-b8bf-e25ae0dd20fc None admin] RBAC: Adding query filter params (role.id=466ed51553a34a62a2c926a8bc552f68, scope.project.id=c02e49a89cdd40d885bfa816f06c58a1, user.id=1f4f49c1b08c460f8ba6c451c9c92c2a) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:23:04.139229 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29c3060-1737-47d8-b8bf-e25ae0dd20fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:04.140659 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29c3060-1737-47d8-b8bf-e25ae0dd20fc None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:04.148405 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 515/1045] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 24 06:23:03 2022] GET /identity/v3/role_assignments?user.id=1f4f49c1b08c460f8ba6c451c9c92c2a&scope.project.id=c02e49a89cdd40d885bfa816f06c58a1&role.id=466ed51553a34a62a2c926a8bc552f68 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:23:04.747866 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed56bc5-89eb-4fb5-b7fc-468f9dad50e9 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:04.748216 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed56bc5-89eb-4fb5-b7fc-468f9dad50e9 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:04.748423 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed56bc5-89eb-4fb5-b7fc-468f9dad50e9 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:04.749842 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 531/1046] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:23:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:23:04.755192 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fb7641-3ab0-447d-b832-7f47b034f43c None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:04.756171 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fb7641-3ab0-447d-b832-7f47b034f43c None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:04.756386 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fb7641-3ab0-447d-b832-7f47b034f43c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:04.775638 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-42fb7641-3ab0-447d-b832-7f47b034f43c None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:23:04.776838 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-42fb7641-3ab0-447d-b832-7f47b034f43c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:23:04.853917 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 516/1047] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:23:04 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:23:05.129503 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed56bc5-89eb-4fb5-b7fc-468f9dad50e9 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:05.142444 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:05.238814 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed56bc5-89eb-4fb5-b7fc-468f9dad50e9 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:05.274610 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a0a8bf-72b7-493d-b9db-46c9cdc95a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:05.275233 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a0a8bf-72b7-493d-b9db-46c9cdc95a7e None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:05.275513 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a0a8bf-72b7-493d-b9db-46c9cdc95a7e None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:05.275786 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a0a8bf-72b7-493d-b9db-46c9cdc95a7e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:05.285814 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a0a8bf-72b7-493d-b9db-46c9cdc95a7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:23:05.291363 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-98a0a8bf-72b7-493d-b9db-46c9cdc95a7e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:23:05.291363 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:23:05.291363 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:23:05.291363 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:23:05.291363 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:23:05.291363 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:23:05.291363 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:23:05.291363 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:23:05.291363 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:23:05.291363 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:23:05.291363 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:23:05.291363 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:23:05.291363 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:23:05.291363 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:23:05.291363 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:23:05.291363 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:23:05.292311 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:23:05.292311 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:23:05.292311 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:23:05.292311 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:23:05.292311 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:23:05.292311 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:23:05.292311 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:23:05.292311 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:23:05.292311 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:23:05.292311 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:23:05.292311 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:23:05.292311 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:23:05.292311 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:23:05.292311 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:23:05.292311 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:23:05.292311 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:23:05.293162 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:23:05.293162 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:23:05.293162 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:23:05.293162 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:23:05.293162 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:23:05.293162 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:23:05.293162 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:23:05.293162 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:23:05.293162 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:23:05.293162 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:23:05.293162 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:23:05.293162 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:23:05.300504 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 532/1048] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Aug 24 06:23:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:23:05.300630 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fb7641-3ab0-447d-b832-7f47b034f43c None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:05.398863 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fb7641-3ab0-447d-b832-7f47b034f43c None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:05.421832 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:05.433704 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f954fb-144c-47b0-8502-d23aa7553f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:05.434308 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f954fb-144c-47b0-8502-d23aa7553f64 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:05.434582 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f954fb-144c-47b0-8502-d23aa7553f64 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:05.434849 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f954fb-144c-47b0-8502-d23aa7553f64 None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:05.435509 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f954fb-144c-47b0-8502-d23aa7553f64 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:23:05.435813 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f954fb-144c-47b0-8502-d23aa7553f64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:05.437151 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f954fb-144c-47b0-8502-d23aa7553f64 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:05.447921 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 517/1049] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 24 06:23:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:23:05.453563 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a0a8bf-72b7-493d-b9db-46c9cdc95a7e None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:05.531283 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:05.571867 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a0a8bf-72b7-493d-b9db-46c9cdc95a7e None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:05.616068 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a4cf67-ddc4-4f6e-bc7f-0500c199feb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:05.616936 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a4cf67-ddc4-4f6e-bc7f-0500c199feb0 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:05.623594 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a4cf67-ddc4-4f6e-bc7f-0500c199feb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:05.623594 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a4cf67-ddc4-4f6e-bc7f-0500c199feb0 None admin] PATH_INFO: `/v3/users/77539e91855944cab4ec716474f59cbd` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:05.646901 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a4cf67-ddc4-4f6e-bc7f-0500c199feb0 None admin] RBAC: Authorizing `identity:get_user(user_id=77539e91855944cab4ec716474f59cbd)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:05.648361 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a4cf67-ddc4-4f6e-bc7f-0500c199feb0 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:05.655019 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 533/1050] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 24 06:23:05 2022] GET /identity/v3/users/77539e91855944cab4ec716474f59cbd => generated 285 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:23:05.656196 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f954fb-144c-47b0-8502-d23aa7553f64 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:05.777022 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f954fb-144c-47b0-8502-d23aa7553f64 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:05.819701 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338b14cd-cc5b-4f76-9561-031d5e174864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:05.820453 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338b14cd-cc5b-4f76-9561-031d5e174864 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:05.820733 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338b14cd-cc5b-4f76-9561-031d5e174864 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:05.821006 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338b14cd-cc5b-4f76-9561-031d5e174864 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:05.881390 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:05.899197 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338b14cd-cc5b-4f76-9561-031d5e174864 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:05.908290 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338b14cd-cc5b-4f76-9561-031d5e174864 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:05.978627 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b3afa0-eb15-4950-809b-8a8892434dd0 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:05.978930 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b3afa0-eb15-4950-809b-8a8892434dd0 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:05.979307 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b3afa0-eb15-4950-809b-8a8892434dd0 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:05.981169 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 534/1051] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:23:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 06:23:05.987176 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd47505f-af21-4814-ab32-57d7356179f8 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:05.987469 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd47505f-af21-4814-ab32-57d7356179f8 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:05.987922 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd47505f-af21-4814-ab32-57d7356179f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:06.006297 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-dd47505f-af21-4814-ab32-57d7356179f8 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:23:06.007514 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-dd47505f-af21-4814-ab32-57d7356179f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:23:06.013264 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 518/1052] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 24 06:23:05 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 06:23:06.016293 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338b14cd-cc5b-4f76-9561-031d5e174864 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:06.082849 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 535/1053] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:23:05 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 06:23:06.085240 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e3f7dd-aff8-421f-b5fe-29d9415597b8 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:06.085698 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e3f7dd-aff8-421f-b5fe-29d9415597b8 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:06.086127 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e3f7dd-aff8-421f-b5fe-29d9415597b8 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:06.087545 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 536/1054] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:23:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:23:06.092897 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa98aad2-13da-4e76-8525-c77c0cc04149 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:06.093166 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa98aad2-13da-4e76-8525-c77c0cc04149 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:06.093468 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa98aad2-13da-4e76-8525-c77c0cc04149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:06.108680 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:06.112980 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-fa98aad2-13da-4e76-8525-c77c0cc04149 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:23:06.114272 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-fa98aad2-13da-4e76-8525-c77c0cc04149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:23:06.124629 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338b14cd-cc5b-4f76-9561-031d5e174864 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:06.164001 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139623de-e9f8-4e6f-8940-44901604cb8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:06.164568 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139623de-e9f8-4e6f-8940-44901604cb8d None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:06.164848 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139623de-e9f8-4e6f-8940-44901604cb8d None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:06.165121 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139623de-e9f8-4e6f-8940-44901604cb8d None admin] PATH_INFO: `/v3/projects/0d963356f36743218a1c2934a7e2a61f` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:06.166288 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139623de-e9f8-4e6f-8940-44901604cb8d None admin] RBAC: Authorizing `identity:get_project(project_id=0d963356f36743218a1c2934a7e2a61f)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:06.167729 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139623de-e9f8-4e6f-8940-44901604cb8d None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:06.169645 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 519/1055] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 24 06:23:06 2022] GET /identity/v3/projects/0d963356f36743218a1c2934a7e2a61f => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:23:06.173873 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:06.175357 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139623de-e9f8-4e6f-8940-44901604cb8d None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:06.192201 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 537/1056] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:23:06 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 06:23:06.280975 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139623de-e9f8-4e6f-8940-44901604cb8d None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:06.315963 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76620fe-5985-41f3-a7c5-fa190ebb135d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:06.316573 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76620fe-5985-41f3-a7c5-fa190ebb135d None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:06.316851 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76620fe-5985-41f3-a7c5-fa190ebb135d None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:06.317122 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76620fe-5985-41f3-a7c5-fa190ebb135d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:06.317911 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76620fe-5985-41f3-a7c5-fa190ebb135d None admin] RBAC: Adding query filter params (role.id=486144ad1bcd417e86acdbc1d14327bf, scope.project.id=0d963356f36743218a1c2934a7e2a61f, user.id=77539e91855944cab4ec716474f59cbd) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:23:06.318246 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76620fe-5985-41f3-a7c5-fa190ebb135d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:06.319668 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76620fe-5985-41f3-a7c5-fa190ebb135d None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:06.333572 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 520/1057] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 24 06:23:06 2022] GET /identity/v3/role_assignments?user.id=77539e91855944cab4ec716474f59cbd&scope.project.id=0d963356f36743218a1c2934a7e2a61f&role.id=486144ad1bcd417e86acdbc1d14327bf => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:23:06.353895 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa98aad2-13da-4e76-8525-c77c0cc04149 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:06.464555 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76620fe-5985-41f3-a7c5-fa190ebb135d None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:06.470438 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa98aad2-13da-4e76-8525-c77c0cc04149 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:06.486986 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:06.515281 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8235faa-daac-49aa-920b-139f8d6cea42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:06.515934 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8235faa-daac-49aa-920b-139f8d6cea42 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:06.516211 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8235faa-daac-49aa-920b-139f8d6cea42 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:06.516504 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8235faa-daac-49aa-920b-139f8d6cea42 None admin] PATH_INFO: `/v3/services/image` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:06.517105 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8235faa-daac-49aa-920b-139f8d6cea42 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:06.518563 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8235faa-daac-49aa-920b-139f8d6cea42 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:06.528236 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-b8235faa-daac-49aa-920b-139f8d6cea42 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 06:23:06.528236 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:23:06.528236 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 06:23:06.528236 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 06:23:06.528236 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 06:23:06.528236 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 06:23:06.528236 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 06:23:06.528236 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:23:06.528236 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 06:23:06.528236 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:23:06.528236 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 06:23:06.528236 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:23:06.528236 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:23:06.528236 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:23:06.528236 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:23:06.528236 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:23:06.528236 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:23:06.529542 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:23:06.529542 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:23:06.529542 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:23:06.529542 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:23:06.529542 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:23:06.529542 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:23:06.529542 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 06:23:06.529542 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 06:23:06.529542 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 06:23:06.529542 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 06:23:06.529542 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:23:06.529542 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:23:06.529542 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:23:06.529542 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:23:06.529542 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:23:06.529542 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:23:06.530579 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:23:06.530579 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:23:06.530579 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:23:06.530579 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:23:06.530579 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:23:06.530579 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:23:06.530579 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:23:06.530579 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:23:06.530579 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:23:06.530579 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:23:06.530579 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 06:23:06.530579 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:23:06.530579 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 06:23:06.530579 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:23:06.530579 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 538/1058] 10.222.0.47 () {56 vars in 1105 bytes} [Wed Aug 24 06:23:06 2022] GET /identity/v3/services/image => generated 86 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:23:06.535242 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8235faa-daac-49aa-920b-139f8d6cea42 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:06.586700 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76620fe-5985-41f3-a7c5-fa190ebb135d None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:06.593426 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:06.633428 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9303cd12-6320-44d3-b0be-7c3499478a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:06.634082 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9303cd12-6320-44d3-b0be-7c3499478a71 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:06.634361 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9303cd12-6320-44d3-b0be-7c3499478a71 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:06.634635 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9303cd12-6320-44d3-b0be-7c3499478a71 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:06.647374 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9303cd12-6320-44d3-b0be-7c3499478a71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:23:06.652138 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-9303cd12-6320-44d3-b0be-7c3499478a71 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 06:23:06.652138 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:23:06.652138 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:23:06.652138 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:23:06.652138 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:23:06.652138 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:23:06.652138 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:23:06.652138 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:23:06.652138 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:23:06.652138 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:23:06.652138 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:23:06.652138 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:23:06.652138 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:23:06.652138 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:23:06.652138 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:23:06.652138 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:23:06.653328 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:23:06.653328 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:23:06.653328 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:23:06.653328 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:23:06.653328 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:23:06.653328 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:23:06.653328 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:23:06.653328 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:23:06.653328 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:23:06.653328 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:23:06.653328 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:23:06.653328 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:23:06.653328 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:23:06.653328 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:23:06.653328 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:23:06.653328 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:23:06.654197 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:23:06.654197 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:23:06.654197 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:23:06.654197 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:23:06.654197 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:23:06.654197 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:23:06.654197 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:23:06.654197 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:23:06.654197 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:23:06.654197 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:23:06.654197 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 06:23:06.654197 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:23:06.654197 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 521/1059] 10.222.0.47 () {56 vars in 1120 bytes} [Wed Aug 24 06:23:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:23:06.659976 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9303cd12-6320-44d3-b0be-7c3499478a71 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:06.675355 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8235faa-daac-49aa-920b-139f8d6cea42 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:06.723600 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9844ed06-34da-48f5-a989-cbf765280128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:06.724742 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9844ed06-34da-48f5-a989-cbf765280128 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:06.724980 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9844ed06-34da-48f5-a989-cbf765280128 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:06.725298 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9844ed06-34da-48f5-a989-cbf765280128 None admin] PATH_INFO: `/v3/services` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:06.725935 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9844ed06-34da-48f5-a989-cbf765280128 None admin] RBAC: Adding query filter params (name=image) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:23:06.726230 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9844ed06-34da-48f5-a989-cbf765280128 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:06.727843 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9844ed06-34da-48f5-a989-cbf765280128 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:06.733625 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 539/1060] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:23:06 2022] GET /identity/v3/services?name=image => generated 122 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:23:06.739078 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9844ed06-34da-48f5-a989-cbf765280128 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:06.747945 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:06.772499 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9303cd12-6320-44d3-b0be-7c3499478a71 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:06.820867 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8569ff-6eb6-4503-a5f9-8673e30fbb69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:06.820867 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8569ff-6eb6-4503-a5f9-8673e30fbb69 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:06.820867 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8569ff-6eb6-4503-a5f9-8673e30fbb69 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:06.820867 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8569ff-6eb6-4503-a5f9-8673e30fbb69 None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:06.821708 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8569ff-6eb6-4503-a5f9-8673e30fbb69 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:23:06.821708 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8569ff-6eb6-4503-a5f9-8673e30fbb69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:06.822460 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8569ff-6eb6-4503-a5f9-8673e30fbb69 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:06.832245 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 522/1061] 10.222.0.47 () {56 vars in 1115 bytes} [Wed Aug 24 06:23:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:23:06.838463 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8569ff-6eb6-4503-a5f9-8673e30fbb69 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:06.868068 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9844ed06-34da-48f5-a989-cbf765280128 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:06.886495 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:06.912835 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79f8609-f26f-4c46-aa5f-b3d0fb8edf62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:06.913525 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79f8609-f26f-4c46-aa5f-b3d0fb8edf62 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:06.913807 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79f8609-f26f-4c46-aa5f-b3d0fb8edf62 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:06.914234 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79f8609-f26f-4c46-aa5f-b3d0fb8edf62 None admin] PATH_INFO: `/v3/services` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:06.915188 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79f8609-f26f-4c46-aa5f-b3d0fb8edf62 None admin] RBAC: Adding query filter params (type=image) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:23:06.915490 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79f8609-f26f-4c46-aa5f-b3d0fb8edf62 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:06.917118 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79f8609-f26f-4c46-aa5f-b3d0fb8edf62 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:06.924086 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 540/1062] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:23:06 2022] GET /identity/v3/services?type=image => generated 350 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:23:06.929344 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79f8609-f26f-4c46-aa5f-b3d0fb8edf62 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:06.960151 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8569ff-6eb6-4503-a5f9-8673e30fbb69 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:06.998966 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208b4d9e-7728-4609-84b4-298e8c5c9c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:06.999521 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208b4d9e-7728-4609-84b4-298e8c5c9c01 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:06.999802 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208b4d9e-7728-4609-84b4-298e8c5c9c01 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:07.000068 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208b4d9e-7728-4609-84b4-298e8c5c9c01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:07.009816 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:07.013999 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208b4d9e-7728-4609-84b4-298e8c5c9c01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:07.015450 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208b4d9e-7728-4609-84b4-298e8c5c9c01 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:07.030995 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-208b4d9e-7728-4609-84b4-298e8c5c9c01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:23:07.030995 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:23:07.030995 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:23:07.030995 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:23:07.030995 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:23:07.030995 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:23:07.030995 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:23:07.030995 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:23:07.030995 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:23:07.030995 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:23:07.030995 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:23:07.030995 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:23:07.030995 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:23:07.030995 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:23:07.030995 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:23:07.030995 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:23:07.032124 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:23:07.032124 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:23:07.032124 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:23:07.032124 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:23:07.032124 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:23:07.032124 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:23:07.032124 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:23:07.032124 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:23:07.032124 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:23:07.032124 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:23:07.032124 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:23:07.032124 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:23:07.032124 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:23:07.032124 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:23:07.032124 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:23:07.032124 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:23:07.032124 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:23:07.033107 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:23:07.033107 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:23:07.033107 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:23:07.033107 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 523/1063] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:23:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:23:07.038204 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208b4d9e-7728-4609-84b4-298e8c5c9c01 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:07.046025 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79f8609-f26f-4c46-aa5f-b3d0fb8edf62 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:07.095730 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbb44fc-fc03-4f88-a54e-4838e4def71d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:07.096772 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbb44fc-fc03-4f88-a54e-4838e4def71d None admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:07.097624 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbb44fc-fc03-4f88-a54e-4838e4def71d None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:07.098340 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbb44fc-fc03-4f88-a54e-4838e4def71d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:07.099586 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbb44fc-fc03-4f88-a54e-4838e4def71d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:07.101822 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbb44fc-fc03-4f88-a54e-4838e4def71d None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:07.153496 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208b4d9e-7728-4609-84b4-298e8c5c9c01 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:07.188315 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 541/1064] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:23:06 2022] POST /identity/v3/endpoints => generated 327 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:23:07.196824 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be0a838-81ed-402d-a340-784d70462574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:07.197489 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be0a838-81ed-402d-a340-784d70462574 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:07.197776 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be0a838-81ed-402d-a340-784d70462574 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:07.198067 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be0a838-81ed-402d-a340-784d70462574 None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:07.198739 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be0a838-81ed-402d-a340-784d70462574 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:23:07.199040 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be0a838-81ed-402d-a340-784d70462574 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:07.200457 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be0a838-81ed-402d-a340-784d70462574 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:07.229147 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 524/1065] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 24 06:23:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:23:07.230475 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbb44fc-fc03-4f88-a54e-4838e4def71d None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:07.281793 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:07.368493 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbb44fc-fc03-4f88-a54e-4838e4def71d None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:07.407324 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d19f035-8821-4e0f-bd34-7297bc5a4ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:07.407999 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d19f035-8821-4e0f-bd34-7297bc5a4ca8 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:07.408281 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d19f035-8821-4e0f-bd34-7297bc5a4ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:07.408554 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d19f035-8821-4e0f-bd34-7297bc5a4ca8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:07.416099 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d19f035-8821-4e0f-bd34-7297bc5a4ca8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:07.417572 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d19f035-8821-4e0f-bd34-7297bc5a4ca8 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:07.431069 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-3d19f035-8821-4e0f-bd34-7297bc5a4ca8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 06:23:07.431069 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:23:07.431069 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:23:07.431069 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:23:07.431069 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:23:07.431069 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:23:07.431069 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:23:07.431069 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:23:07.431069 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:23:07.431069 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:23:07.431069 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:23:07.431069 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:23:07.431069 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 06:23:07.431069 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 06:23:07.431069 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 06:23:07.431069 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 06:23:07.432117 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:23:07.432117 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:23:07.432117 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 06:23:07.432117 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:23:07.432117 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 06:23:07.432117 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:23:07.432117 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:23:07.432117 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:23:07.432117 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:23:07.432117 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:23:07.432117 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:23:07.432117 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:23:07.432117 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:23:07.432117 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:23:07.432117 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:23:07.432117 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:23:07.432117 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:23:07.432117 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:23:07.437923 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:23:07.437923 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:23:07.437923 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 06:23:07.437923 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 06:23:07.437923 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 06:23:07.437923 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 06:23:07.437923 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 06:23:07.437923 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 06:23:07.437923 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 06:23:07.437923 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:23:07.437923 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 542/1066] 10.222.0.47 () {56 vars in 1093 bytes} [Wed Aug 24 06:23:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:23:07.440289 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be0a838-81ed-402d-a340-784d70462574 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:07.501266 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:07.566281 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be0a838-81ed-402d-a340-784d70462574 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:07.604616 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1699a0d8-20c5-4012-a62a-5088f1cd15fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:07.605257 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1699a0d8-20c5-4012-a62a-5088f1cd15fa None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:07.605541 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1699a0d8-20c5-4012-a62a-5088f1cd15fa None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:07.605814 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1699a0d8-20c5-4012-a62a-5088f1cd15fa None admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:07.606551 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1699a0d8-20c5-4012-a62a-5088f1cd15fa None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:23:07.606856 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1699a0d8-20c5-4012-a62a-5088f1cd15fa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:07.608270 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1699a0d8-20c5-4012-a62a-5088f1cd15fa None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:07.634868 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 525/1067] 10.222.0.47 () {56 vars in 1133 bytes} [Wed Aug 24 06:23:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:23:07.641882 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d19f035-8821-4e0f-bd34-7297bc5a4ca8 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:07.749351 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:07.756670 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d19f035-8821-4e0f-bd34-7297bc5a4ca8 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:07.796132 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd153a0d-c660-4678-920d-5af0bd6cec6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:07.798385 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd153a0d-c660-4678-920d-5af0bd6cec6d None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:07.798385 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd153a0d-c660-4678-920d-5af0bd6cec6d None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:07.798385 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd153a0d-c660-4678-920d-5af0bd6cec6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:07.809052 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd153a0d-c660-4678-920d-5af0bd6cec6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:07.810701 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd153a0d-c660-4678-920d-5af0bd6cec6d None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:07.827101 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-cd153a0d-c660-4678-920d-5af0bd6cec6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:23:07.827101 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:23:07.827101 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:23:07.827101 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:23:07.827101 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:23:07.827101 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:23:07.827101 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:23:07.827101 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:23:07.827101 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:23:07.827101 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:23:07.827101 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:23:07.827101 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:23:07.827101 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:23:07.827101 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:23:07.827101 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:23:07.827101 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:23:07.835034 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:23:07.835034 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:23:07.835034 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:23:07.835034 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:23:07.835034 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:23:07.835034 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:23:07.835034 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:23:07.835034 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:23:07.835034 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:23:07.835034 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:23:07.835034 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:23:07.835034 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:23:07.835034 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:23:07.835034 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:23:07.835034 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:23:07.835034 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:23:07.835034 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:23:07.836826 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:23:07.836826 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:23:07.836826 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:23:07.836826 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 543/1068] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:23:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:23:07.843011 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1699a0d8-20c5-4012-a62a-5088f1cd15fa None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:08.012673 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1699a0d8-20c5-4012-a62a-5088f1cd15fa None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:08.016408 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:08.072088 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c5a07c-f042-41a9-8731-35333ca4a6c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:08.073095 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c5a07c-f042-41a9-8731-35333ca4a6c2 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:08.073618 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c5a07c-f042-41a9-8731-35333ca4a6c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:08.074104 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c5a07c-f042-41a9-8731-35333ca4a6c2 None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:08.075240 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c5a07c-f042-41a9-8731-35333ca4a6c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:23:08.075787 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c5a07c-f042-41a9-8731-35333ca4a6c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:08.078423 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c5a07c-f042-41a9-8731-35333ca4a6c2 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:08.096875 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 526/1069] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 24 06:23:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:23:08.103847 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd153a0d-c660-4678-920d-5af0bd6cec6d None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:08.207497 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd153a0d-c660-4678-920d-5af0bd6cec6d None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:08.244288 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3a86e8-d436-44ee-8a87-7f0434e3af8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:08.244921 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3a86e8-d436-44ee-8a87-7f0434e3af8f None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:08.245219 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3a86e8-d436-44ee-8a87-7f0434e3af8f None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:08.245502 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3a86e8-d436-44ee-8a87-7f0434e3af8f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:08.252143 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3a86e8-d436-44ee-8a87-7f0434e3af8f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:08.253605 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3a86e8-d436-44ee-8a87-7f0434e3af8f None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:08.265349 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-4f3a86e8-d436-44ee-8a87-7f0434e3af8f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:23:08.265349 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:23:08.265349 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:23:08.265349 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:23:08.265349 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:23:08.265349 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:23:08.265349 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:23:08.265349 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:23:08.265349 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:23:08.265349 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:23:08.265349 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:23:08.265349 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:23:08.265349 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 06:23:08.265349 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 06:23:08.265349 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 06:23:08.266284 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 06:23:08.266284 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:23:08.266284 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:23:08.266284 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:23:08.266284 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:23:08.266284 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:23:08.266284 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:23:08.266284 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:23:08.266284 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:23:08.266284 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:23:08.266284 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:23:08.266284 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:23:08.266284 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:23:08.266284 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:23:08.266284 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:23:08.266284 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:23:08.266284 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:23:08.267176 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 06:23:08.267176 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 06:23:08.267176 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:23:08.267176 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:23:08.267176 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:23:08.267176 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:23:08.267176 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:23:08.267176 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:23:08.269517 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 544/1070] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 24 06:23:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:23:08.274039 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c5a07c-f042-41a9-8731-35333ca4a6c2 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:08.316774 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31bf68e-5460-434f-823e-4f95ad6e54cb None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:08.317114 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31bf68e-5460-434f-823e-4f95ad6e54cb None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:08.317397 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31bf68e-5460-434f-823e-4f95ad6e54cb None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:08.319081 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 545/1071] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:23:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:23:08.323940 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98ca31d-55f5-4db9-af7c-faf3b68226a1 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:08.324222 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98ca31d-55f5-4db9-af7c-faf3b68226a1 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:08.324497 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98ca31d-55f5-4db9-af7c-faf3b68226a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:08.338550 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:08.343885 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-f98ca31d-55f5-4db9-af7c-faf3b68226a1 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:23:08.353157 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-f98ca31d-55f5-4db9-af7c-faf3b68226a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:23:08.382230 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c5a07c-f042-41a9-8731-35333ca4a6c2 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:08.418439 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 546/1072] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:23:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 06:23:08.422057 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46f8b85-dac5-4235-a6d9-ed8399e25ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:08.422703 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46f8b85-dac5-4235-a6d9-ed8399e25ecc None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:08.422980 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46f8b85-dac5-4235-a6d9-ed8399e25ecc None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:08.423251 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46f8b85-dac5-4235-a6d9-ed8399e25ecc None admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:08.423965 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46f8b85-dac5-4235-a6d9-ed8399e25ecc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:23:08.424269 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46f8b85-dac5-4235-a6d9-ed8399e25ecc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:08.425896 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46f8b85-dac5-4235-a6d9-ed8399e25ecc None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:08.439774 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 527/1073] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Aug 24 06:23:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:23:08.448345 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98ca31d-55f5-4db9-af7c-faf3b68226a1 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:08.549995 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98ca31d-55f5-4db9-af7c-faf3b68226a1 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:08.594582 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78d9e30-d34f-4d0e-b598-20de60d348f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:08.595588 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78d9e30-d34f-4d0e-b598-20de60d348f5 None admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:08.596102 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78d9e30-d34f-4d0e-b598-20de60d348f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:08.596582 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78d9e30-d34f-4d0e-b598-20de60d348f5 None admin] PATH_INFO: `/v3/projects/c02e49a89cdd40d885bfa816f06c58a1/users/1f4f49c1b08c460f8ba6c451c9c92c2a/roles/466ed51553a34a62a2c926a8bc552f68` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:08.599242 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78d9e30-d34f-4d0e-b598-20de60d348f5 None admin] RBAC: Authorizing `identity:create_grant(project_id=c02e49a89cdd40d885bfa816f06c58a1, user_id=1f4f49c1b08c460f8ba6c451c9c92c2a, role_id=466ed51553a34a62a2c926a8bc552f68)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:08.600905 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78d9e30-d34f-4d0e-b598-20de60d348f5 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:08.604276 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:08.684392 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 547/1074] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Aug 24 06:23:08 2022] PUT /identity/v3/projects/c02e49a89cdd40d885bfa816f06c58a1/users/1f4f49c1b08c460f8ba6c451c9c92c2a/roles/466ed51553a34a62a2c926a8bc552f68 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:23:08.702854 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46f8b85-dac5-4235-a6d9-ed8399e25ecc None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:08.732731 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:08.838267 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46f8b85-dac5-4235-a6d9-ed8399e25ecc None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:08.876402 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2b5c4e-501c-4eeb-afc2-488869e1cdaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:08.877429 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b5c4e-501c-4eeb-afc2-488869e1cdaa None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:08.877726 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b5c4e-501c-4eeb-afc2-488869e1cdaa None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:08.878106 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b5c4e-501c-4eeb-afc2-488869e1cdaa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:08.892043 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2b5c4e-501c-4eeb-afc2-488869e1cdaa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:23:08.897292 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-1b2b5c4e-501c-4eeb-afc2-488869e1cdaa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:23:08.897292 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:23:08.897292 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:23:08.897292 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:23:08.897292 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:23:08.897292 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:23:08.897292 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:23:08.897292 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:23:08.897292 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:23:08.897292 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:23:08.897292 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:23:08.897292 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:23:08.897292 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:23:08.897292 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:23:08.897292 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:23:08.897292 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:23:08.898517 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:23:08.898517 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:23:08.898517 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:23:08.898517 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:23:08.898517 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:23:08.898517 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:23:08.898517 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:23:08.898517 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:23:08.898517 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:23:08.898517 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:23:08.898517 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:23:08.898517 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:23:08.898517 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:23:08.898517 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:23:08.898517 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:23:08.898517 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:23:08.902270 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:23:08.902270 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:23:08.902270 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:23:08.902270 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:23:08.902270 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:23:08.902270 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:23:08.902270 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:23:08.902270 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:23:08.902270 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:23:08.902270 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:23:08.902270 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:23:08.902270 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:23:08.902270 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 528/1075] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Aug 24 06:23:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:23:08.905925 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78d9e30-d34f-4d0e-b598-20de60d348f5 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:09.021618 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78d9e30-d34f-4d0e-b598-20de60d348f5 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:09.066241 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36b13b8-9803-426d-b95c-3bcd62026569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:09.067125 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36b13b8-9803-426d-b95c-3bcd62026569 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:09.068144 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36b13b8-9803-426d-b95c-3bcd62026569 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:09.068551 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36b13b8-9803-426d-b95c-3bcd62026569 None admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:09.069594 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36b13b8-9803-426d-b95c-3bcd62026569 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:23:09.070048 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36b13b8-9803-426d-b95c-3bcd62026569 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:09.072481 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36b13b8-9803-426d-b95c-3bcd62026569 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:09.091306 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 548/1076] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 24 06:23:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:23:09.093130 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2b5c4e-501c-4eeb-afc2-488869e1cdaa None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:09.208718 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:09.226409 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2b5c4e-501c-4eeb-afc2-488869e1cdaa None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:09.267794 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc4b8fb-291d-43e9-8ffd-29d1cbce3cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:09.269129 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc4b8fb-291d-43e9-8ffd-29d1cbce3cd5 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:09.269646 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc4b8fb-291d-43e9-8ffd-29d1cbce3cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:09.270140 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc4b8fb-291d-43e9-8ffd-29d1cbce3cd5 None admin] PATH_INFO: `/v3/users/77539e91855944cab4ec716474f59cbd` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:09.272336 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc4b8fb-291d-43e9-8ffd-29d1cbce3cd5 None admin] RBAC: Authorizing `identity:get_user(user_id=77539e91855944cab4ec716474f59cbd)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:09.274030 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc4b8fb-291d-43e9-8ffd-29d1cbce3cd5 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:09.280435 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 529/1077] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 24 06:23:09 2022] GET /identity/v3/users/77539e91855944cab4ec716474f59cbd => generated 285 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:23:09.282209 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36b13b8-9803-426d-b95c-3bcd62026569 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:09.295400 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80583ecd-6d87-4ad1-a07b-e5ef7bc3a0f2 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:09.295710 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80583ecd-6d87-4ad1-a07b-e5ef7bc3a0f2 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:09.295970 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80583ecd-6d87-4ad1-a07b-e5ef7bc3a0f2 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:09.297366 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 530/1078] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:23:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:23:09.303840 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6348310d-6301-44e4-88a9-74e47e2bbf75 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:09.304126 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6348310d-6301-44e4-88a9-74e47e2bbf75 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:09.304372 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6348310d-6301-44e4-88a9-74e47e2bbf75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:09.323989 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-6348310d-6301-44e4-88a9-74e47e2bbf75 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:23:09.325230 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-6348310d-6301-44e4-88a9-74e47e2bbf75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:23:09.351380 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:09.398278 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36b13b8-9803-426d-b95c-3bcd62026569 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:09.399122 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 531/1079] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:23:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 06:23:09.437657 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ef516a-f4d2-4fc6-9a27-cdbbe33a6c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:09.438293 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ef516a-f4d2-4fc6-9a27-cdbbe33a6c86 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:09.438568 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ef516a-f4d2-4fc6-9a27-cdbbe33a6c86 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:09.438836 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ef516a-f4d2-4fc6-9a27-cdbbe33a6c86 None admin] PATH_INFO: `/v3/projects/0d963356f36743218a1c2934a7e2a61f` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:09.439963 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ef516a-f4d2-4fc6-9a27-cdbbe33a6c86 None admin] RBAC: Authorizing `identity:get_project(project_id=0d963356f36743218a1c2934a7e2a61f)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:09.441424 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ef516a-f4d2-4fc6-9a27-cdbbe33a6c86 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:09.448197 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 549/1080] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 24 06:23:09 2022] GET /identity/v3/projects/0d963356f36743218a1c2934a7e2a61f => generated 311 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:23:09.449253 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6348310d-6301-44e4-88a9-74e47e2bbf75 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:09.533982 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:09.554104 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6348310d-6301-44e4-88a9-74e47e2bbf75 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:09.600543 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1779dd-6208-41d3-bf14-6e9b8cb8e2f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:09.601295 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1779dd-6208-41d3-bf14-6e9b8cb8e2f8 None admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:09.601582 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1779dd-6208-41d3-bf14-6e9b8cb8e2f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:09.601861 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1779dd-6208-41d3-bf14-6e9b8cb8e2f8 None admin] PATH_INFO: `/v3/projects/0d963356f36743218a1c2934a7e2a61f/users/77539e91855944cab4ec716474f59cbd/roles/486144ad1bcd417e86acdbc1d14327bf` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:09.603806 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1779dd-6208-41d3-bf14-6e9b8cb8e2f8 None admin] RBAC: Authorizing `identity:create_grant(project_id=0d963356f36743218a1c2934a7e2a61f, user_id=77539e91855944cab4ec716474f59cbd, role_id=486144ad1bcd417e86acdbc1d14327bf)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:09.605274 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1779dd-6208-41d3-bf14-6e9b8cb8e2f8 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:09.676160 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ef516a-f4d2-4fc6-9a27-cdbbe33a6c86 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:09.720507 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 532/1081] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Aug 24 06:23:09 2022] PUT /identity/v3/projects/0d963356f36743218a1c2934a7e2a61f/users/77539e91855944cab4ec716474f59cbd/roles/486144ad1bcd417e86acdbc1d14327bf => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:23:09.797366 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ef516a-f4d2-4fc6-9a27-cdbbe33a6c86 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:09.827818 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:09.842603 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d77b6e1-d36b-4b5c-8b27-4669fd4b4513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:09.843564 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d77b6e1-d36b-4b5c-8b27-4669fd4b4513 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:09.844032 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d77b6e1-d36b-4b5c-8b27-4669fd4b4513 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:09.844490 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d77b6e1-d36b-4b5c-8b27-4669fd4b4513 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:09.924638 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d77b6e1-d36b-4b5c-8b27-4669fd4b4513 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:09.926128 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d77b6e1-d36b-4b5c-8b27-4669fd4b4513 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:10.036656 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 550/1082] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 24 06:23:09 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 06:23:10.042684 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1779dd-6208-41d3-bf14-6e9b8cb8e2f8 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:10.151207 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1779dd-6208-41d3-bf14-6e9b8cb8e2f8 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:10.187105 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d63bb4-ff4a-4963-9efe-76e2656b35f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:10.187746 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d63bb4-ff4a-4963-9efe-76e2656b35f4 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:10.188023 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d63bb4-ff4a-4963-9efe-76e2656b35f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:10.188290 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d63bb4-ff4a-4963-9efe-76e2656b35f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:10.201437 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d63bb4-ff4a-4963-9efe-76e2656b35f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:10.202874 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d63bb4-ff4a-4963-9efe-76e2656b35f4 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:10.204660 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:10.219099 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-b4d63bb4-ff4a-4963-9efe-76e2656b35f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:23:10.219099 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:23:10.219099 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:23:10.219099 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:23:10.219099 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:23:10.219099 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:23:10.219099 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:23:10.219099 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:23:10.219099 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:23:10.219099 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:23:10.219099 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:23:10.219099 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:23:10.219099 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:23:10.219099 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:23:10.219099 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:23:10.219099 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:23:10.220190 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:23:10.220190 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:23:10.220190 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:23:10.220190 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:23:10.220190 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:23:10.220190 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:23:10.220190 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:23:10.220190 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:23:10.220190 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:23:10.220190 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:23:10.220190 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:23:10.220190 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:23:10.220190 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:23:10.220190 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:23:10.220190 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:23:10.220190 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:23:10.220190 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:23:10.221120 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:23:10.221120 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:23:10.221120 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:23:10.221996 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 533/1083] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:23:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:23:10.226465 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d77b6e1-d36b-4b5c-8b27-4669fd4b4513 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:10.255497 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:10.343401 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d77b6e1-d36b-4b5c-8b27-4669fd4b4513 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:10.391472 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174218f5-30c0-4ed5-ade8-9bf9ea298d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:10.392065 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174218f5-30c0-4ed5-ade8-9bf9ea298d7f None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:10.392346 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174218f5-30c0-4ed5-ade8-9bf9ea298d7f None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:10.392616 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174218f5-30c0-4ed5-ade8-9bf9ea298d7f None admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:10.393304 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174218f5-30c0-4ed5-ade8-9bf9ea298d7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:23:10.393863 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174218f5-30c0-4ed5-ade8-9bf9ea298d7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:10.395230 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174218f5-30c0-4ed5-ade8-9bf9ea298d7f None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:10.417512 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 551/1084] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 24 06:23:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:23:10.928745 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c1af4e-e878-4686-a3ce-8199c7c93106 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:10.929156 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c1af4e-e878-4686-a3ce-8199c7c93106 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:10.929649 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c1af4e-e878-4686-a3ce-8199c7c93106 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:10.931190 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 534/1085] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:23:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:23:10.936606 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49d2ec2-5f6b-425d-b339-1b3dc9fe7ca9 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:10.936891 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49d2ec2-5f6b-425d-b339-1b3dc9fe7ca9 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:10.937169 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49d2ec2-5f6b-425d-b339-1b3dc9fe7ca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:10.956248 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-e49d2ec2-5f6b-425d-b339-1b3dc9fe7ca9 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:23:10.957595 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-e49d2ec2-5f6b-425d-b339-1b3dc9fe7ca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:23:11.027735 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 552/1086] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:23:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:23:11.300223 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c1af4e-e878-4686-a3ce-8199c7c93106 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:11.407950 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c1af4e-e878-4686-a3ce-8199c7c93106 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:11.443367 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927ef734-784d-4a46-b765-5087ff9ecf2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:11.444003 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927ef734-784d-4a46-b765-5087ff9ecf2c None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:11.444283 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927ef734-784d-4a46-b765-5087ff9ecf2c None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:11.444556 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927ef734-784d-4a46-b765-5087ff9ecf2c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:11.454652 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927ef734-784d-4a46-b765-5087ff9ecf2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:23:11.459539 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-927ef734-784d-4a46-b765-5087ff9ecf2c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 06:23:11.459539 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:23:11.459539 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:23:11.459539 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:23:11.459539 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:23:11.459539 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:23:11.459539 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:23:11.459539 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:23:11.459539 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:23:11.459539 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:23:11.459539 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:23:11.459539 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:23:11.459539 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:23:11.459539 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:23:11.459539 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:23:11.459539 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:23:11.460661 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:23:11.460661 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:23:11.460661 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:23:11.460661 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:23:11.460661 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:23:11.460661 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:23:11.460661 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:23:11.460661 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:23:11.460661 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:23:11.460661 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:23:11.460661 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:23:11.460661 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:23:11.460661 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:23:11.460661 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:23:11.460661 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:23:11.460661 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:23:11.461731 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:23:11.461731 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:23:11.461731 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:23:11.461731 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:23:11.461731 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:23:11.461731 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:23:11.461731 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:23:11.461731 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:23:11.461731 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:23:11.461731 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:23:11.461731 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 06:23:11.461731 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:23:11.461731 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 535/1087] 10.222.0.47 () {56 vars in 1120 bytes} [Wed Aug 24 06:23:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:23:11.466796 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49d2ec2-5f6b-425d-b339-1b3dc9fe7ca9 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:11.487156 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:11.571646 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49d2ec2-5f6b-425d-b339-1b3dc9fe7ca9 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:11.605460 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a50f74-551a-43d0-a9ba-24ed1ca3e9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:11.606071 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a50f74-551a-43d0-a9ba-24ed1ca3e9a9 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:11.606352 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a50f74-551a-43d0-a9ba-24ed1ca3e9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:11.606627 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a50f74-551a-43d0-a9ba-24ed1ca3e9a9 None admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:11.607292 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a50f74-551a-43d0-a9ba-24ed1ca3e9a9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:23:11.607594 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a50f74-551a-43d0-a9ba-24ed1ca3e9a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:11.608954 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a50f74-551a-43d0-a9ba-24ed1ca3e9a9 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:11.620258 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 553/1088] 10.222.0.47 () {56 vars in 1115 bytes} [Wed Aug 24 06:23:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:23:11.623513 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927ef734-784d-4a46-b765-5087ff9ecf2c None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:11.665920 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc1f93d-8698-4738-9fc0-566da05c18a4 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:11.666213 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc1f93d-8698-4738-9fc0-566da05c18a4 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:11.666485 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc1f93d-8698-4738-9fc0-566da05c18a4 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:11.667821 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 554/1089] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:23:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:23:11.672760 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e48d5b-25e8-4fc1-a9da-93325f6e94bd None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:11.673035 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e48d5b-25e8-4fc1-a9da-93325f6e94bd None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:11.673340 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e48d5b-25e8-4fc1-a9da-93325f6e94bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:11.681414 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:11.691672 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-37e48d5b-25e8-4fc1-a9da-93325f6e94bd None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:23:11.693095 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-37e48d5b-25e8-4fc1-a9da-93325f6e94bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:23:11.728085 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927ef734-784d-4a46-b765-5087ff9ecf2c None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:11.764570 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 555/1090] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:23:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 06:23:11.765828 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b2ef27-93a8-4a92-9e71-ce0e09988003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:11.766647 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b2ef27-93a8-4a92-9e71-ce0e09988003 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:11.766920 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b2ef27-93a8-4a92-9e71-ce0e09988003 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:11.767187 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b2ef27-93a8-4a92-9e71-ce0e09988003 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:11.778525 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b2ef27-93a8-4a92-9e71-ce0e09988003 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:11.780082 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b2ef27-93a8-4a92-9e71-ce0e09988003 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:11.794797 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-a1b2ef27-93a8-4a92-9e71-ce0e09988003 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:23:11.794797 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:23:11.794797 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:23:11.794797 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:23:11.794797 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:23:11.794797 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:23:11.794797 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:23:11.794797 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:23:11.794797 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:23:11.794797 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:23:11.794797 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:23:11.794797 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:23:11.794797 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:23:11.794797 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:23:11.794797 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:23:11.794797 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:23:11.795878 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:23:11.795878 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:23:11.795878 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:23:11.795878 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:23:11.795878 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:23:11.795878 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:23:11.795878 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:23:11.795878 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:23:11.795878 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:23:11.795878 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:23:11.795878 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:23:11.795878 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:23:11.795878 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:23:11.795878 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:23:11.795878 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:23:11.795878 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:23:11.795878 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:23:11.796800 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:23:11.796800 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:23:11.796800 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:23:11.796800 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 536/1091] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:23:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:23:11.803136 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e48d5b-25e8-4fc1-a9da-93325f6e94bd None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:11.813820 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:11.909081 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e48d5b-25e8-4fc1-a9da-93325f6e94bd None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:11.944006 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ddaea7-3a4e-46ff-87e6-0d6812bcff0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:11.944613 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ddaea7-3a4e-46ff-87e6-0d6812bcff0a None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:11.944896 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ddaea7-3a4e-46ff-87e6-0d6812bcff0a None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:11.945193 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ddaea7-3a4e-46ff-87e6-0d6812bcff0a None admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:11.945842 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ddaea7-3a4e-46ff-87e6-0d6812bcff0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:23:11.946143 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ddaea7-3a4e-46ff-87e6-0d6812bcff0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:11.947511 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ddaea7-3a4e-46ff-87e6-0d6812bcff0a None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:11.959563 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 556/1092] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 24 06:23:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:23:11.965312 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b2ef27-93a8-4a92-9e71-ce0e09988003 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:12.038208 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ddaea7-3a4e-46ff-87e6-0d6812bcff0a None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:12.070808 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b2ef27-93a8-4a92-9e71-ce0e09988003 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:12.078509 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:12.105372 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:12.106278 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab02ffe5-7751-4582-bdf8-bdaf70e8a523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:12.106915 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab02ffe5-7751-4582-bdf8-bdaf70e8a523 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:12.107191 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab02ffe5-7751-4582-bdf8-bdaf70e8a523 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:12.107476 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab02ffe5-7751-4582-bdf8-bdaf70e8a523 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:12.116767 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab02ffe5-7751-4582-bdf8-bdaf70e8a523 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:12.118252 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab02ffe5-7751-4582-bdf8-bdaf70e8a523 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:12.130406 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-ab02ffe5-7751-4582-bdf8-bdaf70e8a523 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 06:23:12.130406 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:23:12.130406 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:23:12.130406 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:23:12.130406 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:23:12.130406 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:23:12.130406 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:23:12.130406 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:23:12.130406 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:23:12.130406 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:23:12.130406 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:23:12.130406 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:23:12.130406 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 06:23:12.130406 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 06:23:12.130406 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 06:23:12.130406 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 06:23:12.131412 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:23:12.131412 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:23:12.131412 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 06:23:12.131412 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:23:12.131412 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 06:23:12.131412 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:23:12.131412 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:23:12.131412 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:23:12.131412 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:23:12.131412 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:23:12.131412 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:23:12.131412 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:23:12.131412 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:23:12.131412 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:23:12.131412 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:23:12.131412 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:23:12.131412 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:23:12.131412 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:23:12.132888 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:23:12.132888 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:23:12.132888 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 06:23:12.132888 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 06:23:12.132888 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 06:23:12.132888 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 06:23:12.132888 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 06:23:12.132888 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 06:23:12.132888 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 06:23:12.132888 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:23:12.132888 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 537/1093] 10.222.0.47 () {56 vars in 1093 bytes} [Wed Aug 24 06:23:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:23:12.137373 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab02ffe5-7751-4582-bdf8-bdaf70e8a523 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:12.139670 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ddaea7-3a4e-46ff-87e6-0d6812bcff0a None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:12.175653 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ed096a-1583-4e5d-93cc-8b7c836544a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:12.176210 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ed096a-1583-4e5d-93cc-8b7c836544a9 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:12.176570 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ed096a-1583-4e5d-93cc-8b7c836544a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:12.176910 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ed096a-1583-4e5d-93cc-8b7c836544a9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:12.186971 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ed096a-1583-4e5d-93cc-8b7c836544a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:23:12.192684 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-b4ed096a-1583-4e5d-93cc-8b7c836544a9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:23:12.192684 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:23:12.192684 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:23:12.192684 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:23:12.192684 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:23:12.192684 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:23:12.192684 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:23:12.192684 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:23:12.192684 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:23:12.192684 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:23:12.192684 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:23:12.192684 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:23:12.192684 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:23:12.192684 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:23:12.192684 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:23:12.192684 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:23:12.193680 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:23:12.193680 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:23:12.193680 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:23:12.193680 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:23:12.193680 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:23:12.193680 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:23:12.193680 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:23:12.193680 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:23:12.193680 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:23:12.193680 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:23:12.193680 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:23:12.193680 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:23:12.193680 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:23:12.193680 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:23:12.193680 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:23:12.193680 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:23:12.194802 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:23:12.194802 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:23:12.194802 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:23:12.194802 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:23:12.194802 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:23:12.194802 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:23:12.194802 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:23:12.194802 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:23:12.194802 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:23:12.194802 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:23:12.194802 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:23:12.194802 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:23:12.194802 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 557/1094] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Aug 24 06:23:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:23:12.199857 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ed096a-1583-4e5d-93cc-8b7c836544a9 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:12.240597 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab02ffe5-7751-4582-bdf8-bdaf70e8a523 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:12.280521 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b88547c-0643-482f-96fc-1a5357c8c37f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:12.281158 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b88547c-0643-482f-96fc-1a5357c8c37f None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:12.281466 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b88547c-0643-482f-96fc-1a5357c8c37f None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:12.281733 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b88547c-0643-482f-96fc-1a5357c8c37f None admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:12.282460 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b88547c-0643-482f-96fc-1a5357c8c37f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:23:12.282761 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b88547c-0643-482f-96fc-1a5357c8c37f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:12.284171 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b88547c-0643-482f-96fc-1a5357c8c37f None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:12.300626 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ed096a-1583-4e5d-93cc-8b7c836544a9 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:12.301662 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 538/1095] 10.222.0.47 () {56 vars in 1133 bytes} [Wed Aug 24 06:23:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:23:12.309223 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b88547c-0643-482f-96fc-1a5357c8c37f None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:12.332297 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:12.335669 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:12.342694 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fd3141-0d8b-4ed0-9733-b2364cfb73a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:12.350434 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fd3141-0d8b-4ed0-9733-b2364cfb73a2 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:12.350434 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fd3141-0d8b-4ed0-9733-b2364cfb73a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:12.350434 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fd3141-0d8b-4ed0-9733-b2364cfb73a2 None admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:12.350434 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fd3141-0d8b-4ed0-9733-b2364cfb73a2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:23:12.350434 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fd3141-0d8b-4ed0-9733-b2364cfb73a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:12.350434 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fd3141-0d8b-4ed0-9733-b2364cfb73a2 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:12.356167 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 558/1096] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 24 06:23:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:23:12.361996 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fd3141-0d8b-4ed0-9733-b2364cfb73a2 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:12.416806 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b88547c-0643-482f-96fc-1a5357c8c37f None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:12.451003 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09057326-448d-4f5e-859a-52ee20968bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:12.451517 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09057326-448d-4f5e-859a-52ee20968bfb None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:12.451806 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09057326-448d-4f5e-859a-52ee20968bfb None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:12.452081 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09057326-448d-4f5e-859a-52ee20968bfb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:12.475814 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fd3141-0d8b-4ed0-9733-b2364cfb73a2 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:12.517092 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df54a86-0751-4dd2-992e-4452078790ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:12.518115 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df54a86-0751-4dd2-992e-4452078790ae None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:12.518742 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df54a86-0751-4dd2-992e-4452078790ae None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:12.519112 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df54a86-0751-4dd2-992e-4452078790ae None admin] PATH_INFO: `/v3/users/77539e91855944cab4ec716474f59cbd` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:12.521542 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df54a86-0751-4dd2-992e-4452078790ae None admin] RBAC: Authorizing `identity:get_user(user_id=77539e91855944cab4ec716474f59cbd)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:12.523484 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df54a86-0751-4dd2-992e-4452078790ae None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:12.525698 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 559/1097] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 24 06:23:12 2022] GET /identity/v3/users/77539e91855944cab4ec716474f59cbd => generated 285 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:23:12.534553 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df54a86-0751-4dd2-992e-4452078790ae None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:12.536574 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09057326-448d-4f5e-859a-52ee20968bfb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:12.538084 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09057326-448d-4f5e-859a-52ee20968bfb None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:12.565890 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:12.597643 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:12.646153 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 539/1098] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 24 06:23:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 06:23:12.646457 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df54a86-0751-4dd2-992e-4452078790ae None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:12.652113 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09057326-448d-4f5e-859a-52ee20968bfb None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:12.681534 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4af6997-37af-4f7d-bd11-e6a8639cf38b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:12.682152 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af6997-37af-4f7d-bd11-e6a8639cf38b None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:12.682432 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af6997-37af-4f7d-bd11-e6a8639cf38b None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:12.682702 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af6997-37af-4f7d-bd11-e6a8639cf38b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:12.754137 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4af6997-37af-4f7d-bd11-e6a8639cf38b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:12.756045 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4af6997-37af-4f7d-bd11-e6a8639cf38b None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:12.757454 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09057326-448d-4f5e-859a-52ee20968bfb None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:12.784343 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:12.795026 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c099159-c60a-4581-9b5a-6b54f3853c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:12.795659 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c099159-c60a-4581-9b5a-6b54f3853c8e None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:12.795945 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c099159-c60a-4581-9b5a-6b54f3853c8e None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:12.796219 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c099159-c60a-4581-9b5a-6b54f3853c8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:12.807068 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c099159-c60a-4581-9b5a-6b54f3853c8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:12.808571 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c099159-c60a-4581-9b5a-6b54f3853c8e None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:12.815208 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:12.824614 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-1c099159-c60a-4581-9b5a-6b54f3853c8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:23:12.824614 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:23:12.824614 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:23:12.824614 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:23:12.824614 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:23:12.824614 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:23:12.824614 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:23:12.824614 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:23:12.824614 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:23:12.824614 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:23:12.824614 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:23:12.824614 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:23:12.824614 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:23:12.824614 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:23:12.824614 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:23:12.824614 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:23:12.825698 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:23:12.825698 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:23:12.825698 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:23:12.825698 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:23:12.825698 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:23:12.825698 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:23:12.825698 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:23:12.825698 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:23:12.825698 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:23:12.825698 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:23:12.825698 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:23:12.825698 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:23:12.825698 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:23:12.825698 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:23:12.825698 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:23:12.825698 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:23:12.825698 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:23:12.826646 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:23:12.826646 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:23:12.826646 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:23:12.830740 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 540/1099] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:23:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:23:12.831659 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c099159-c60a-4581-9b5a-6b54f3853c8e None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:12.860243 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 560/1100] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 24 06:23:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 06:23:12.866479 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4af6997-37af-4f7d-bd11-e6a8639cf38b None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:12.935660 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c099159-c60a-4581-9b5a-6b54f3853c8e None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:12.970998 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4af6997-37af-4f7d-bd11-e6a8639cf38b None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:12.971900 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cc0e43-af1d-43fa-838a-37446bce5b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:12.972523 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cc0e43-af1d-43fa-838a-37446bce5b5e None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:12.972803 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cc0e43-af1d-43fa-838a-37446bce5b5e None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:12.973085 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cc0e43-af1d-43fa-838a-37446bce5b5e None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:12.973791 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cc0e43-af1d-43fa-838a-37446bce5b5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:23:12.974085 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cc0e43-af1d-43fa-838a-37446bce5b5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:12.975448 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cc0e43-af1d-43fa-838a-37446bce5b5e None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:12.987684 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 541/1101] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 24 06:23:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:23:12.993745 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cc0e43-af1d-43fa-838a-37446bce5b5e None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:13.005601 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d565db60-8185-416d-89e5-e734516136cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:13.006233 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d565db60-8185-416d-89e5-e734516136cb None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:13.006528 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d565db60-8185-416d-89e5-e734516136cb None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:13.006801 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d565db60-8185-416d-89e5-e734516136cb None admin] PATH_INFO: `/v3/projects/0d963356f36743218a1c2934a7e2a61f` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:13.007921 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d565db60-8185-416d-89e5-e734516136cb None admin] RBAC: Authorizing `identity:get_project(project_id=0d963356f36743218a1c2934a7e2a61f)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:13.009368 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d565db60-8185-416d-89e5-e734516136cb None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:13.011157 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 561/1102] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 24 06:23:12 2022] GET /identity/v3/projects/0d963356f36743218a1c2934a7e2a61f => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:23:13.017045 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d565db60-8185-416d-89e5-e734516136cb None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:13.029392 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:13.045735 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:13.100763 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cc0e43-af1d-43fa-838a-37446bce5b5e None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:13.121075 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d565db60-8185-416d-89e5-e734516136cb None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:13.135397 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0f81c1-956b-4237-b21e-6de27c11c72f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:13.136040 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0f81c1-956b-4237-b21e-6de27c11c72f None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:13.136320 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0f81c1-956b-4237-b21e-6de27c11c72f None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:13.136594 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0f81c1-956b-4237-b21e-6de27c11c72f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:13.143578 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0f81c1-956b-4237-b21e-6de27c11c72f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:13.145050 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0f81c1-956b-4237-b21e-6de27c11c72f None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:13.155585 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea251d06-347e-4bb7-8651-ac26875c2252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:13.156245 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea251d06-347e-4bb7-8651-ac26875c2252 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:13.156465 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-bc0f81c1-956b-4237-b21e-6de27c11c72f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:23:13.156465 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:23:13.156465 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:23:13.156465 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:23:13.156465 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:23:13.156465 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:23:13.156465 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:23:13.156465 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:23:13.156465 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:23:13.156465 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:23:13.156465 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:23:13.156465 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:23:13.156465 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 06:23:13.156465 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 06:23:13.156465 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 06:23:13.157478 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 06:23:13.157478 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:23:13.157478 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:23:13.157478 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:23:13.157478 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:23:13.157478 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:23:13.157478 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:23:13.157478 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:23:13.157478 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:23:13.157478 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:23:13.157478 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:23:13.157478 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:23:13.157478 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:23:13.157478 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:23:13.157478 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:23:13.157478 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:23:13.157478 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:23:13.158574 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 06:23:13.158574 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 06:23:13.158574 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:23:13.158574 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:23:13.158574 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:23:13.158574 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:23:13.158574 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:23:13.158574 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:23:13.159177 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea251d06-347e-4bb7-8651-ac26875c2252 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:13.159177 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea251d06-347e-4bb7-8651-ac26875c2252 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:13.159177 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea251d06-347e-4bb7-8651-ac26875c2252 None admin] RBAC: Adding query filter params (role.id=486144ad1bcd417e86acdbc1d14327bf, scope.project.id=0d963356f36743218a1c2934a7e2a61f, user.id=77539e91855944cab4ec716474f59cbd) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:23:13.159391 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 542/1103] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 24 06:23:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:23:13.161509 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea251d06-347e-4bb7-8651-ac26875c2252 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:13.163034 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea251d06-347e-4bb7-8651-ac26875c2252 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:13.164697 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0f81c1-956b-4237-b21e-6de27c11c72f None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:13.175793 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 562/1104] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 24 06:23:13 2022] GET /identity/v3/role_assignments?user.id=77539e91855944cab4ec716474f59cbd&scope.project.id=0d963356f36743218a1c2934a7e2a61f&role.id=486144ad1bcd417e86acdbc1d14327bf => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:23:13.269166 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:13.275287 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0f81c1-956b-4237-b21e-6de27c11c72f None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:13.312515 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed718c95-9f6d-4e99-865f-79cfe56de0fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:13.313229 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed718c95-9f6d-4e99-865f-79cfe56de0fe None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:13.313504 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed718c95-9f6d-4e99-865f-79cfe56de0fe None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:13.313773 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed718c95-9f6d-4e99-865f-79cfe56de0fe None admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:13.314473 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed718c95-9f6d-4e99-865f-79cfe56de0fe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:23:13.314775 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed718c95-9f6d-4e99-865f-79cfe56de0fe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:13.316195 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed718c95-9f6d-4e99-865f-79cfe56de0fe None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:13.331575 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 543/1105] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Aug 24 06:23:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:23:13.338147 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea251d06-347e-4bb7-8651-ac26875c2252 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:13.433767 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:13.443307 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea251d06-347e-4bb7-8651-ac26875c2252 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:13.478594 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bce38ed-665b-4371-9f7c-ad43587a8c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:13.479231 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bce38ed-665b-4371-9f7c-ad43587a8c08 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:13.479455 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bce38ed-665b-4371-9f7c-ad43587a8c08 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:13.479780 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bce38ed-665b-4371-9f7c-ad43587a8c08 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:13.480502 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bce38ed-665b-4371-9f7c-ad43587a8c08 None admin] RBAC: Adding query filter params (role.id=466ed51553a34a62a2c926a8bc552f68, scope.project.id=c02e49a89cdd40d885bfa816f06c58a1, user.id=1f4f49c1b08c460f8ba6c451c9c92c2a) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:23:13.480804 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bce38ed-665b-4371-9f7c-ad43587a8c08 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:13.482257 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bce38ed-665b-4371-9f7c-ad43587a8c08 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:13.488163 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 563/1106] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 24 06:23:13 2022] GET /identity/v3/role_assignments?user.id=1f4f49c1b08c460f8ba6c451c9c92c2a&scope.project.id=c02e49a89cdd40d885bfa816f06c58a1&role.id=466ed51553a34a62a2c926a8bc552f68 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:23:15.200247 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfb685c-a8d2-4516-a911-15c77819b981 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:15.200781 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfb685c-a8d2-4516-a911-15c77819b981 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:15.200781 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfb685c-a8d2-4516-a911-15c77819b981 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:15.202150 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 544/1107] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:23:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:23:15.207330 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70475f4d-36c8-46c4-87c5-e7eb52fbe3b5 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:15.208339 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70475f4d-36c8-46c4-87c5-e7eb52fbe3b5 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:15.208473 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70475f4d-36c8-46c4-87c5-e7eb52fbe3b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:15.227204 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-70475f4d-36c8-46c4-87c5-e7eb52fbe3b5 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:23:15.228432 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-70475f4d-36c8-46c4-87c5-e7eb52fbe3b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:23:15.296030 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 564/1108] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:23:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:23:15.564163 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfb685c-a8d2-4516-a911-15c77819b981 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:15.665576 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfb685c-a8d2-4516-a911-15c77819b981 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:15.698873 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6d6b93-0164-467d-99b7-e673fe09fa3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:15.699500 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6d6b93-0164-467d-99b7-e673fe09fa3d None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:15.699780 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6d6b93-0164-467d-99b7-e673fe09fa3d None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:15.700050 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6d6b93-0164-467d-99b7-e673fe09fa3d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:15.701160 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6d6b93-0164-467d-99b7-e673fe09fa3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:15.702889 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6d6b93-0164-467d-99b7-e673fe09fa3d None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:15.704172 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 545/1109] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:23:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:23:15.710141 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70475f4d-36c8-46c4-87c5-e7eb52fbe3b5 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:15.792178 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:15.807819 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70475f4d-36c8-46c4-87c5-e7eb52fbe3b5 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:15.842194 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5aab8b-ab0b-4b46-bb28-0a2b655ae76e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:15.842721 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5aab8b-ab0b-4b46-bb28-0a2b655ae76e None admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:15.843001 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5aab8b-ab0b-4b46-bb28-0a2b655ae76e None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:15.843271 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5aab8b-ab0b-4b46-bb28-0a2b655ae76e None admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:15.843924 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5aab8b-ab0b-4b46-bb28-0a2b655ae76e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:15.845389 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5aab8b-ab0b-4b46-bb28-0a2b655ae76e None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:15.958538 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 565/1110] 10.222.0.47 () {60 vars in 1132 bytes} [Wed Aug 24 06:23:15 2022] POST /identity/v3/users => generated 286 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:23:17.983557 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb7c21d-2b96-427a-abf8-313edc5c24b8 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:17.984079 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb7c21d-2b96-427a-abf8-313edc5c24b8 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:17.984179 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb7c21d-2b96-427a-abf8-313edc5c24b8 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:17.985545 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 546/1111] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:23:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:23:17.992061 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8147d2-770c-408e-bbb0-f737efcf329a None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:17.992531 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8147d2-770c-408e-bbb0-f737efcf329a None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:17.992622 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8147d2-770c-408e-bbb0-f737efcf329a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:18.011326 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-1e8147d2-770c-408e-bbb0-f737efcf329a None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:23:18.012522 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-1e8147d2-770c-408e-bbb0-f737efcf329a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:23:18.082446 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 566/1112] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:23:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:23:18.353482 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb7c21d-2b96-427a-abf8-313edc5c24b8 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:18.378782 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:18.452969 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb7c21d-2b96-427a-abf8-313edc5c24b8 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:18.485637 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce75039c-0d45-4b25-be8f-cf3f728f0ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:18.486257 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce75039c-0d45-4b25-be8f-cf3f728f0ffc None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:18.486532 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce75039c-0d45-4b25-be8f-cf3f728f0ffc None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:18.486802 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce75039c-0d45-4b25-be8f-cf3f728f0ffc None admin] PATH_INFO: `/v3/roles/b71c87e12dc1452ea3c99a7bf28585ca` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:18.488166 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce75039c-0d45-4b25-be8f-cf3f728f0ffc None admin] RBAC: Authorizing `identity:get_role(role_id=b71c87e12dc1452ea3c99a7bf28585ca)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:18.490619 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce75039c-0d45-4b25-be8f-cf3f728f0ffc None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:18.491927 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 547/1113] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 24 06:23:18 2022] GET /identity/v3/roles/b71c87e12dc1452ea3c99a7bf28585ca => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:23:18.497867 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8147d2-770c-408e-bbb0-f737efcf329a None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:18.565010 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:18.594509 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8147d2-770c-408e-bbb0-f737efcf329a None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:18.627790 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ef70c8-5edd-4f39-8e32-c76f0c26707b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:18.628420 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ef70c8-5edd-4f39-8e32-c76f0c26707b None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:18.628725 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ef70c8-5edd-4f39-8e32-c76f0c26707b None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:18.629082 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ef70c8-5edd-4f39-8e32-c76f0c26707b None admin] PATH_INFO: `/v3/users/c0eb7c2d323a40a481748dc356b3ea62` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:18.642703 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ef70c8-5edd-4f39-8e32-c76f0c26707b None admin] RBAC: Authorizing `identity:get_user(user_id=c0eb7c2d323a40a481748dc356b3ea62)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:18.644399 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ef70c8-5edd-4f39-8e32-c76f0c26707b None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:18.646836 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 567/1114] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 24 06:23:18 2022] GET /identity/v3/users/c0eb7c2d323a40a481748dc356b3ea62 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:23:18.652504 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce75039c-0d45-4b25-be8f-cf3f728f0ffc None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:18.740574 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:18.748872 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce75039c-0d45-4b25-be8f-cf3f728f0ffc None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:18.783107 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd22a17-cb69-4202-a6b3-ce515bfcf9ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:18.783712 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd22a17-cb69-4202-a6b3-ce515bfcf9ce None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:18.783991 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd22a17-cb69-4202-a6b3-ce515bfcf9ce None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:18.784261 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd22a17-cb69-4202-a6b3-ce515bfcf9ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:18.852227 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd22a17-cb69-4202-a6b3-ce515bfcf9ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:18.853817 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd22a17-cb69-4202-a6b3-ce515bfcf9ce None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:18.945902 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:18.955382 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 548/1115] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 24 06:23:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 06:23:18.962138 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ef70c8-5edd-4f39-8e32-c76f0c26707b None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:19.069079 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ef70c8-5edd-4f39-8e32-c76f0c26707b None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:19.097677 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:19.106693 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ac62c1-7e5d-4c13-b433-d46c99349e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:19.107337 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ac62c1-7e5d-4c13-b433-d46c99349e81 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:19.107619 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ac62c1-7e5d-4c13-b433-d46c99349e81 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:19.107894 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ac62c1-7e5d-4c13-b433-d46c99349e81 None admin] PATH_INFO: `/v3/projects/0d963356f36743218a1c2934a7e2a61f` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:19.109021 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ac62c1-7e5d-4c13-b433-d46c99349e81 None admin] RBAC: Authorizing `identity:get_project(project_id=0d963356f36743218a1c2934a7e2a61f)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:19.110521 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ac62c1-7e5d-4c13-b433-d46c99349e81 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:19.112170 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 568/1116] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 24 06:23:18 2022] GET /identity/v3/projects/0d963356f36743218a1c2934a7e2a61f => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:23:19.118474 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd22a17-cb69-4202-a6b3-ce515bfcf9ce None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:19.219671 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd22a17-cb69-4202-a6b3-ce515bfcf9ce None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:19.256040 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b2f16f-be3f-42be-a648-6150407ab268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:19.256649 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b2f16f-be3f-42be-a648-6150407ab268 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:19.256930 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b2f16f-be3f-42be-a648-6150407ab268 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:19.257222 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b2f16f-be3f-42be-a648-6150407ab268 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:19.257965 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b2f16f-be3f-42be-a648-6150407ab268 None admin] RBAC: Adding query filter params (role.id=b71c87e12dc1452ea3c99a7bf28585ca, scope.project.id=0d963356f36743218a1c2934a7e2a61f, user.id=c0eb7c2d323a40a481748dc356b3ea62) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:23:19.258273 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b2f16f-be3f-42be-a648-6150407ab268 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:19.259696 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b2f16f-be3f-42be-a648-6150407ab268 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:19.265394 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 549/1117] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 24 06:23:19 2022] GET /identity/v3/role_assignments?user.id=c0eb7c2d323a40a481748dc356b3ea62&scope.project.id=0d963356f36743218a1c2934a7e2a61f&role.id=b71c87e12dc1452ea3c99a7bf28585ca => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:23:21.154860 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df689688-c7b5-45bd-9e26-b51cf4cb1c7a None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:21.155411 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df689688-c7b5-45bd-9e26-b51cf4cb1c7a None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:21.155411 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df689688-c7b5-45bd-9e26-b51cf4cb1c7a None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:21.156732 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 569/1118] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:23:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:23:21.162506 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b1cb79-85f6-4748-a575-ae83026b2fb3 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:21.162918 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b1cb79-85f6-4748-a575-ae83026b2fb3 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:21.163050 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b1cb79-85f6-4748-a575-ae83026b2fb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:21.180574 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-a7b1cb79-85f6-4748-a575-ae83026b2fb3 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:23:21.181787 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-a7b1cb79-85f6-4748-a575-ae83026b2fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:23:21.230816 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:21.246547 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 550/1119] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:23:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:23:21.516830 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df689688-c7b5-45bd-9e26-b51cf4cb1c7a None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:21.623778 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df689688-c7b5-45bd-9e26-b51cf4cb1c7a None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:21.658414 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc795f7-2ffd-4263-b848-673bedbdc48e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:21.659041 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc795f7-2ffd-4263-b848-673bedbdc48e None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:21.659325 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc795f7-2ffd-4263-b848-673bedbdc48e None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:21.659598 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc795f7-2ffd-4263-b848-673bedbdc48e None admin] PATH_INFO: `/v3/roles/b71c87e12dc1452ea3c99a7bf28585ca` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:21.661400 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc795f7-2ffd-4263-b848-673bedbdc48e None admin] RBAC: Authorizing `identity:get_role(role_id=b71c87e12dc1452ea3c99a7bf28585ca)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:21.662755 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc795f7-2ffd-4263-b848-673bedbdc48e None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:21.664076 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 570/1120] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 24 06:23:21 2022] GET /identity/v3/roles/b71c87e12dc1452ea3c99a7bf28585ca => generated 223 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:23:21.670069 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b1cb79-85f6-4748-a575-ae83026b2fb3 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:21.765556 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b1cb79-85f6-4748-a575-ae83026b2fb3 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:21.800475 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dab6cd-d89e-409b-a53a-662dadd08c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:21.801064 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dab6cd-d89e-409b-a53a-662dadd08c75 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:21.801376 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dab6cd-d89e-409b-a53a-662dadd08c75 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:21.801657 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dab6cd-d89e-409b-a53a-662dadd08c75 None admin] PATH_INFO: `/v3/users/c0eb7c2d323a40a481748dc356b3ea62` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:21.802791 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dab6cd-d89e-409b-a53a-662dadd08c75 None admin] RBAC: Authorizing `identity:get_user(user_id=c0eb7c2d323a40a481748dc356b3ea62)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:21.804214 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dab6cd-d89e-409b-a53a-662dadd08c75 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:21.805828 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 551/1121] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 24 06:23:21 2022] GET /identity/v3/users/c0eb7c2d323a40a481748dc356b3ea62 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:23:21.811716 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc795f7-2ffd-4263-b848-673bedbdc48e None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:21.877552 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:21.919814 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc795f7-2ffd-4263-b848-673bedbdc48e None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:21.957654 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b31586-e497-4a1a-a2fc-9387f719d80f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:21.958376 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b31586-e497-4a1a-a2fc-9387f719d80f None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:21.958652 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b31586-e497-4a1a-a2fc-9387f719d80f None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:21.958927 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b31586-e497-4a1a-a2fc-9387f719d80f None admin] PATH_INFO: `/v3/projects/0d963356f36743218a1c2934a7e2a61f` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:21.960079 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b31586-e497-4a1a-a2fc-9387f719d80f None admin] RBAC: Authorizing `identity:get_project(project_id=0d963356f36743218a1c2934a7e2a61f)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:21.961532 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b31586-e497-4a1a-a2fc-9387f719d80f None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:21.963163 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 571/1122] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 24 06:23:21 2022] GET /identity/v3/projects/0d963356f36743218a1c2934a7e2a61f => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:23:21.969099 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dab6cd-d89e-409b-a53a-662dadd08c75 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:22.024054 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:22.065962 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dab6cd-d89e-409b-a53a-662dadd08c75 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:22.099360 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee1d0cf-694a-45d7-b997-ba3c137a78df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:22.100017 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee1d0cf-694a-45d7-b997-ba3c137a78df None admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:22.100300 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee1d0cf-694a-45d7-b997-ba3c137a78df None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:22.100579 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee1d0cf-694a-45d7-b997-ba3c137a78df None admin] PATH_INFO: `/v3/projects/0d963356f36743218a1c2934a7e2a61f/users/c0eb7c2d323a40a481748dc356b3ea62/roles/b71c87e12dc1452ea3c99a7bf28585ca` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:22.102774 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee1d0cf-694a-45d7-b997-ba3c137a78df None admin] RBAC: Authorizing `identity:create_grant(project_id=0d963356f36743218a1c2934a7e2a61f, user_id=c0eb7c2d323a40a481748dc356b3ea62, role_id=b71c87e12dc1452ea3c99a7bf28585ca)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:22.104200 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee1d0cf-694a-45d7-b997-ba3c137a78df None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:22.156814 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 552/1123] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Aug 24 06:23:21 2022] PUT /identity/v3/projects/0d963356f36743218a1c2934a7e2a61f/users/c0eb7c2d323a40a481748dc356b3ea62/roles/b71c87e12dc1452ea3c99a7bf28585ca => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:23:24.044211 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd35ef0-c033-4744-8bff-59792e0b9ec2 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:24.044734 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd35ef0-c033-4744-8bff-59792e0b9ec2 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:24.044734 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd35ef0-c033-4744-8bff-59792e0b9ec2 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:24.046032 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 572/1124] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:23:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:23:24.051696 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef6f967-feab-4e5b-b8ba-e4890466d96b None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:24.052084 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef6f967-feab-4e5b-b8ba-e4890466d96b None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:24.052242 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef6f967-feab-4e5b-b8ba-e4890466d96b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:24.071270 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-4ef6f967-feab-4e5b-b8ba-e4890466d96b None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:23:24.072453 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-4ef6f967-feab-4e5b-b8ba-e4890466d96b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:23:24.138455 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 553/1125] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:23:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:23:24.423790 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd35ef0-c033-4744-8bff-59792e0b9ec2 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:24.525454 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd35ef0-c033-4744-8bff-59792e0b9ec2 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:24.555568 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:24.560241 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e639a95a-0c23-4038-9db8-b71e14f6fdbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:24.560867 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e639a95a-0c23-4038-9db8-b71e14f6fdbd None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:24.561144 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e639a95a-0c23-4038-9db8-b71e14f6fdbd None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:24.561447 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e639a95a-0c23-4038-9db8-b71e14f6fdbd None admin] PATH_INFO: `/v3/roles/b71c87e12dc1452ea3c99a7bf28585ca` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:24.562835 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e639a95a-0c23-4038-9db8-b71e14f6fdbd None admin] RBAC: Authorizing `identity:get_role(role_id=b71c87e12dc1452ea3c99a7bf28585ca)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:24.564191 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e639a95a-0c23-4038-9db8-b71e14f6fdbd None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:24.565554 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 573/1126] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 24 06:23:24 2022] GET /identity/v3/roles/b71c87e12dc1452ea3c99a7bf28585ca => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:23:24.575626 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef6f967-feab-4e5b-b8ba-e4890466d96b None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:24.616356 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:24.673977 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef6f967-feab-4e5b-b8ba-e4890466d96b None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:24.707556 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab83a943-6f56-45ad-981f-a3adfbd3829d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:24.708191 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab83a943-6f56-45ad-981f-a3adfbd3829d None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:24.708474 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab83a943-6f56-45ad-981f-a3adfbd3829d None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:24.708753 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab83a943-6f56-45ad-981f-a3adfbd3829d None admin] PATH_INFO: `/v3/users/c0eb7c2d323a40a481748dc356b3ea62` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:24.709893 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab83a943-6f56-45ad-981f-a3adfbd3829d None admin] RBAC: Authorizing `identity:get_user(user_id=c0eb7c2d323a40a481748dc356b3ea62)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:24.711323 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab83a943-6f56-45ad-981f-a3adfbd3829d None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:24.712929 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 554/1127] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 24 06:23:24 2022] GET /identity/v3/users/c0eb7c2d323a40a481748dc356b3ea62 => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:23:24.718940 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e639a95a-0c23-4038-9db8-b71e14f6fdbd None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:24.818577 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e639a95a-0c23-4038-9db8-b71e14f6fdbd None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:24.852660 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0b209e-024c-407d-8249-9a26e36f145d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:24.853286 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b209e-024c-407d-8249-9a26e36f145d None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:24.853571 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b209e-024c-407d-8249-9a26e36f145d None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:24.853849 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b209e-024c-407d-8249-9a26e36f145d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:24.917998 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:24.920136 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0b209e-024c-407d-8249-9a26e36f145d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:24.921663 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0b209e-024c-407d-8249-9a26e36f145d None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:25.024649 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 574/1128] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 24 06:23:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 06:23:25.030612 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab83a943-6f56-45ad-981f-a3adfbd3829d None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:25.128892 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab83a943-6f56-45ad-981f-a3adfbd3829d None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:25.143053 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:25.165079 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9c3c1f-5c0d-4109-8eb3-f534a2f25016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:25.165733 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c3c1f-5c0d-4109-8eb3-f534a2f25016 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:25.166012 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c3c1f-5c0d-4109-8eb3-f534a2f25016 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:25.166281 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c3c1f-5c0d-4109-8eb3-f534a2f25016 None admin] PATH_INFO: `/v3/projects/0d963356f36743218a1c2934a7e2a61f` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:25.167386 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9c3c1f-5c0d-4109-8eb3-f534a2f25016 None admin] RBAC: Authorizing `identity:get_project(project_id=0d963356f36743218a1c2934a7e2a61f)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:25.168793 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9c3c1f-5c0d-4109-8eb3-f534a2f25016 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:25.170433 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 555/1129] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 24 06:23:25 2022] GET /identity/v3/projects/0d963356f36743218a1c2934a7e2a61f => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:23:25.177588 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0b209e-024c-407d-8249-9a26e36f145d None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:25.273772 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:25.276734 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0b209e-024c-407d-8249-9a26e36f145d None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:25.314519 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37da496-ac87-45f6-b88f-01181067183c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:25.315121 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37da496-ac87-45f6-b88f-01181067183c None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:25.315397 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37da496-ac87-45f6-b88f-01181067183c None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:25.315668 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37da496-ac87-45f6-b88f-01181067183c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:25.316445 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37da496-ac87-45f6-b88f-01181067183c None admin] RBAC: Adding query filter params (role.id=b71c87e12dc1452ea3c99a7bf28585ca, scope.project.id=0d963356f36743218a1c2934a7e2a61f, user.id=c0eb7c2d323a40a481748dc356b3ea62) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:23:25.316747 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37da496-ac87-45f6-b88f-01181067183c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:25.318201 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37da496-ac87-45f6-b88f-01181067183c None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:25.323930 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 575/1130] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 24 06:23:25 2022] GET /identity/v3/role_assignments?user.id=c0eb7c2d323a40a481748dc356b3ea62&scope.project.id=0d963356f36743218a1c2934a7e2a61f&role.id=b71c87e12dc1452ea3c99a7bf28585ca => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:23:27.263049 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0006e236-4a20-4670-8ed4-1922aab85fbc None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:27.263580 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0006e236-4a20-4670-8ed4-1922aab85fbc None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:27.263580 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0006e236-4a20-4670-8ed4-1922aab85fbc None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:27.264935 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 556/1131] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:23:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:23:27.271205 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc740c3d-721c-48da-8121-b5392ff83008 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:27.271628 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc740c3d-721c-48da-8121-b5392ff83008 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:27.271746 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc740c3d-721c-48da-8121-b5392ff83008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:27.289607 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-cc740c3d-721c-48da-8121-b5392ff83008 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:23:27.290801 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-cc740c3d-721c-48da-8121-b5392ff83008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:23:27.361169 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 576/1132] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:23:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:23:27.630220 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0006e236-4a20-4670-8ed4-1922aab85fbc None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:27.745988 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0006e236-4a20-4670-8ed4-1922aab85fbc None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:27.779070 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f14ebc-343c-4b45-9c55-0b5d4989988e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:27.779693 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f14ebc-343c-4b45-9c55-0b5d4989988e None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:27.779973 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f14ebc-343c-4b45-9c55-0b5d4989988e None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:27.780252 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f14ebc-343c-4b45-9c55-0b5d4989988e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:27.781383 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f14ebc-343c-4b45-9c55-0b5d4989988e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:27.782798 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f14ebc-343c-4b45-9c55-0b5d4989988e None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:27.784405 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 557/1133] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:23:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:23:27.791213 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc740c3d-721c-48da-8121-b5392ff83008 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:27.883909 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:27.890315 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc740c3d-721c-48da-8121-b5392ff83008 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:27.924793 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c1e9a5-1823-403f-9d91-9758af84c914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:27.927204 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c1e9a5-1823-403f-9d91-9758af84c914 None admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:27.927204 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c1e9a5-1823-403f-9d91-9758af84c914 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:27.927204 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c1e9a5-1823-403f-9d91-9758af84c914 None admin] PATH_INFO: `/v3/projects` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:27.927204 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c1e9a5-1823-403f-9d91-9758af84c914 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:27.928311 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c1e9a5-1823-403f-9d91-9758af84c914 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:28.009896 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 577/1134] 10.222.0.47 () {60 vars in 1141 bytes} [Wed Aug 24 06:23:27 2022] POST /identity/v3/projects => generated 311 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:23:30.117705 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcbb68c-c200-4304-8220-4f29147a95ce None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:30.118225 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcbb68c-c200-4304-8220-4f29147a95ce None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:30.118225 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcbb68c-c200-4304-8220-4f29147a95ce None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:30.119572 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 558/1135] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:23:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:23:30.124895 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca23df94-2994-4711-9bb6-fcc7803874cd None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:30.125311 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca23df94-2994-4711-9bb6-fcc7803874cd None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:30.125486 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca23df94-2994-4711-9bb6-fcc7803874cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:30.144101 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-ca23df94-2994-4711-9bb6-fcc7803874cd None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:23:30.145338 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-ca23df94-2994-4711-9bb6-fcc7803874cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:23:30.212088 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 578/1136] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:23:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:23:30.481327 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcbb68c-c200-4304-8220-4f29147a95ce None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:30.538258 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:30.591826 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcbb68c-c200-4304-8220-4f29147a95ce None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:30.625197 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9f18e7-dcfc-4cd2-94a2-b9575c94557e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:30.625835 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9f18e7-dcfc-4cd2-94a2-b9575c94557e None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:30.626114 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9f18e7-dcfc-4cd2-94a2-b9575c94557e None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:30.626388 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9f18e7-dcfc-4cd2-94a2-b9575c94557e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:30.627491 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9f18e7-dcfc-4cd2-94a2-b9575c94557e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:30.629618 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9f18e7-dcfc-4cd2-94a2-b9575c94557e None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:30.631174 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 559/1137] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:23:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:23:30.636912 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca23df94-2994-4711-9bb6-fcc7803874cd None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:30.711823 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:30.734392 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca23df94-2994-4711-9bb6-fcc7803874cd None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:30.769862 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321f066f-f1e0-43a8-9854-db9b266e4a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:30.770459 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321f066f-f1e0-43a8-9854-db9b266e4a01 None admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:30.770740 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321f066f-f1e0-43a8-9854-db9b266e4a01 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:30.771021 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321f066f-f1e0-43a8-9854-db9b266e4a01 None admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:30.771657 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321f066f-f1e0-43a8-9854-db9b266e4a01 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:30.773094 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321f066f-f1e0-43a8-9854-db9b266e4a01 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:30.842483 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 579/1138] 10.222.0.47 () {60 vars in 1132 bytes} [Wed Aug 24 06:23:30 2022] POST /identity/v3/users => generated 286 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:23:32.781943 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4843dc-4fff-4a65-8660-c968dd1d54cd None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:32.782464 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4843dc-4fff-4a65-8660-c968dd1d54cd None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:32.782464 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4843dc-4fff-4a65-8660-c968dd1d54cd None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:32.783803 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 560/1139] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:23:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:23:32.789452 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f451c01-a388-4727-b030-12382ef38f6f None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:32.790527 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f451c01-a388-4727-b030-12382ef38f6f None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:32.790527 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f451c01-a388-4727-b030-12382ef38f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:32.807996 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-2f451c01-a388-4727-b030-12382ef38f6f None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:23:32.809256 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-2f451c01-a388-4727-b030-12382ef38f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:23:32.876968 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 580/1140] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:23:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:23:33.147316 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4843dc-4fff-4a65-8660-c968dd1d54cd None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:33.245896 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4843dc-4fff-4a65-8660-c968dd1d54cd None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:33.261782 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:33.278903 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25faa14f-936e-4a15-9a2c-ee4e9da51ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:33.279525 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25faa14f-936e-4a15-9a2c-ee4e9da51ca3 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:33.279801 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25faa14f-936e-4a15-9a2c-ee4e9da51ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:33.280079 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25faa14f-936e-4a15-9a2c-ee4e9da51ca3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:33.290678 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25faa14f-936e-4a15-9a2c-ee4e9da51ca3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:23:33.295334 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-25faa14f-936e-4a15-9a2c-ee4e9da51ca3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:23:33.295334 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:23:33.295334 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:23:33.295334 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:23:33.295334 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:23:33.295334 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:23:33.295334 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:23:33.295334 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:23:33.295334 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:23:33.295334 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:23:33.295334 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:23:33.295334 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:23:33.295334 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:23:33.295334 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:23:33.295334 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:23:33.295334 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:23:33.296547 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:23:33.296547 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:23:33.296547 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:23:33.296547 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:23:33.296547 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:23:33.296547 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:23:33.296547 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:23:33.296547 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:23:33.296547 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:23:33.296547 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:23:33.296547 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:23:33.296547 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:23:33.296547 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:23:33.296547 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:23:33.296547 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:23:33.296547 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:23:33.297657 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:23:33.297657 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:23:33.297657 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:23:33.297657 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:23:33.297657 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:23:33.297657 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:23:33.297657 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:23:33.297657 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:23:33.297657 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:23:33.297657 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:23:33.297657 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:23:33.297657 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:23:33.297657 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 561/1141] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Aug 24 06:23:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:23:33.305447 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f451c01-a388-4727-b030-12382ef38f6f None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:33.366628 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:33.408267 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f451c01-a388-4727-b030-12382ef38f6f None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:33.444699 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530dbaf4-891d-49b2-9aa9-c7bc92f009f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:33.445343 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530dbaf4-891d-49b2-9aa9-c7bc92f009f9 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:33.445628 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530dbaf4-891d-49b2-9aa9-c7bc92f009f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:33.445904 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530dbaf4-891d-49b2-9aa9-c7bc92f009f9 None admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:33.446576 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530dbaf4-891d-49b2-9aa9-c7bc92f009f9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:23:33.446882 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530dbaf4-891d-49b2-9aa9-c7bc92f009f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:33.448258 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530dbaf4-891d-49b2-9aa9-c7bc92f009f9 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:33.456583 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 581/1142] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 24 06:23:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:23:33.462400 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25faa14f-936e-4a15-9a2c-ee4e9da51ca3 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:33.562549 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25faa14f-936e-4a15-9a2c-ee4e9da51ca3 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:33.597155 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29932045-24ae-4b02-877b-91a9d857cccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:33.597843 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29932045-24ae-4b02-877b-91a9d857cccb None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:33.598126 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29932045-24ae-4b02-877b-91a9d857cccb None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:33.598396 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29932045-24ae-4b02-877b-91a9d857cccb None admin] PATH_INFO: `/v3/users/08da66fcd3e040c581997eab611095bb` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:33.617076 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29932045-24ae-4b02-877b-91a9d857cccb None admin] RBAC: Authorizing `identity:get_user(user_id=08da66fcd3e040c581997eab611095bb)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:33.618587 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29932045-24ae-4b02-877b-91a9d857cccb None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:33.620634 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 562/1143] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 24 06:23:33 2022] GET /identity/v3/users/08da66fcd3e040c581997eab611095bb => generated 286 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:23:33.626269 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530dbaf4-891d-49b2-9aa9-c7bc92f009f9 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:33.727908 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530dbaf4-891d-49b2-9aa9-c7bc92f009f9 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:33.757715 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:33.769213 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c174e5-e10c-49d9-bef5-5a604548d09b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:33.770035 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c174e5-e10c-49d9-bef5-5a604548d09b None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:33.770312 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c174e5-e10c-49d9-bef5-5a604548d09b None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:33.770581 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c174e5-e10c-49d9-bef5-5a604548d09b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:33.838656 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c174e5-e10c-49d9-bef5-5a604548d09b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:33.840122 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c174e5-e10c-49d9-bef5-5a604548d09b None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:33.942036 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 582/1144] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 24 06:23:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 06:23:33.947987 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29932045-24ae-4b02-877b-91a9d857cccb None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:33.981751 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:34.047765 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29932045-24ae-4b02-877b-91a9d857cccb None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:34.082782 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11513d39-c30d-4806-a294-9838bb11daaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:34.083401 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11513d39-c30d-4806-a294-9838bb11daaf None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:34.083699 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11513d39-c30d-4806-a294-9838bb11daaf None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:34.083966 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11513d39-c30d-4806-a294-9838bb11daaf None admin] PATH_INFO: `/v3/projects/6ab53ff2383b487face7ccf9bf1fbecb` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:34.085063 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11513d39-c30d-4806-a294-9838bb11daaf None admin] RBAC: Authorizing `identity:get_project(project_id=6ab53ff2383b487face7ccf9bf1fbecb)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:34.086499 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11513d39-c30d-4806-a294-9838bb11daaf None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:34.088107 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 563/1145] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 24 06:23:33 2022] GET /identity/v3/projects/6ab53ff2383b487face7ccf9bf1fbecb => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:23:34.094297 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c174e5-e10c-49d9-bef5-5a604548d09b None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:34.126877 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:34.201987 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c174e5-e10c-49d9-bef5-5a604548d09b None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:34.236289 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75bf0d0-4e45-4166-9061-6962e7ba3317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:34.237003 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75bf0d0-4e45-4166-9061-6962e7ba3317 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:34.237842 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75bf0d0-4e45-4166-9061-6962e7ba3317 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:34.238118 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75bf0d0-4e45-4166-9061-6962e7ba3317 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:34.238842 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75bf0d0-4e45-4166-9061-6962e7ba3317 None admin] RBAC: Adding query filter params (role.id=486144ad1bcd417e86acdbc1d14327bf, scope.project.id=6ab53ff2383b487face7ccf9bf1fbecb, user.id=08da66fcd3e040c581997eab611095bb) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:23:34.239150 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75bf0d0-4e45-4166-9061-6962e7ba3317 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:34.240579 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75bf0d0-4e45-4166-9061-6962e7ba3317 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:34.245858 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 583/1146] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 24 06:23:34 2022] GET /identity/v3/role_assignments?user.id=08da66fcd3e040c581997eab611095bb&scope.project.id=6ab53ff2383b487face7ccf9bf1fbecb&role.id=486144ad1bcd417e86acdbc1d14327bf => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:23:36.116108 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33c983b-9334-4c86-b7c5-4752959a1c16 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:36.116620 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33c983b-9334-4c86-b7c5-4752959a1c16 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:36.116620 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33c983b-9334-4c86-b7c5-4752959a1c16 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:36.117999 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 564/1147] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:23:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:23:36.123231 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c345c026-47bf-42f3-a26b-ac27a0cd2ffd None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:36.124402 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c345c026-47bf-42f3-a26b-ac27a0cd2ffd None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:36.124402 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c345c026-47bf-42f3-a26b-ac27a0cd2ffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:36.142476 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-c345c026-47bf-42f3-a26b-ac27a0cd2ffd None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:23:36.144443 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-c345c026-47bf-42f3-a26b-ac27a0cd2ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:23:36.210065 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 584/1148] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:23:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:23:36.489333 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33c983b-9334-4c86-b7c5-4752959a1c16 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:36.588429 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33c983b-9334-4c86-b7c5-4752959a1c16 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:36.589949 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:36.622508 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f3480e-f392-46a5-9024-5af6cb6ad882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:36.622733 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f3480e-f392-46a5-9024-5af6cb6ad882 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:36.623014 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f3480e-f392-46a5-9024-5af6cb6ad882 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:36.623288 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f3480e-f392-46a5-9024-5af6cb6ad882 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:36.634391 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f3480e-f392-46a5-9024-5af6cb6ad882 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:23:36.639042 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-78f3480e-f392-46a5-9024-5af6cb6ad882 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:23:36.639042 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:23:36.639042 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:23:36.639042 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:23:36.639042 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:23:36.639042 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:23:36.639042 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:23:36.639042 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:23:36.639042 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:23:36.639042 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:23:36.639042 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:23:36.639042 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:23:36.639042 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:23:36.639042 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:23:36.639042 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:23:36.639042 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:23:36.640143 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:23:36.640143 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:23:36.640143 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:23:36.640143 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:23:36.640143 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:23:36.640143 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:23:36.640143 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:23:36.640143 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:23:36.640143 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:23:36.640143 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:23:36.640143 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:23:36.640143 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:23:36.640143 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:23:36.640143 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:23:36.640143 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:23:36.640143 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:23:36.640986 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:23:36.640986 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:23:36.640986 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:23:36.640986 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:23:36.640986 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:23:36.640986 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:23:36.640986 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:23:36.640986 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:23:36.640986 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:23:36.640986 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:23:36.640986 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:23:36.640986 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:23:36.640986 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 565/1149] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Aug 24 06:23:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:23:36.646485 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c345c026-47bf-42f3-a26b-ac27a0cd2ffd None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:36.664493 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:36.743877 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c345c026-47bf-42f3-a26b-ac27a0cd2ffd None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:36.776861 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1cebcb-9810-4f26-81c4-7d1e5ceef075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:36.777480 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1cebcb-9810-4f26-81c4-7d1e5ceef075 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:36.777755 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1cebcb-9810-4f26-81c4-7d1e5ceef075 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:36.778019 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1cebcb-9810-4f26-81c4-7d1e5ceef075 None admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:36.778668 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1cebcb-9810-4f26-81c4-7d1e5ceef075 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:23:36.778963 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1cebcb-9810-4f26-81c4-7d1e5ceef075 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:36.780317 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1cebcb-9810-4f26-81c4-7d1e5ceef075 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:36.788382 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 585/1150] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 24 06:23:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:23:36.794374 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f3480e-f392-46a5-9024-5af6cb6ad882 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:36.985921 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f3480e-f392-46a5-9024-5af6cb6ad882 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:37.022813 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7452b08d-90a9-46fe-ae96-b66072b37885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:37.023363 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7452b08d-90a9-46fe-ae96-b66072b37885 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:37.023639 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7452b08d-90a9-46fe-ae96-b66072b37885 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:37.023971 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7452b08d-90a9-46fe-ae96-b66072b37885 None admin] PATH_INFO: `/v3/users/08da66fcd3e040c581997eab611095bb` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:37.025113 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7452b08d-90a9-46fe-ae96-b66072b37885 None admin] RBAC: Authorizing `identity:get_user(user_id=08da66fcd3e040c581997eab611095bb)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:37.026562 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7452b08d-90a9-46fe-ae96-b66072b37885 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:37.028137 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 566/1151] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 24 06:23:36 2022] GET /identity/v3/users/08da66fcd3e040c581997eab611095bb => generated 286 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:23:37.034205 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1cebcb-9810-4f26-81c4-7d1e5ceef075 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:37.119297 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:37.139262 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1cebcb-9810-4f26-81c4-7d1e5ceef075 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:37.176780 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7e7cce-d907-4574-8623-69d0ececc6a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:37.177439 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7e7cce-d907-4574-8623-69d0ececc6a2 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:37.177721 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7e7cce-d907-4574-8623-69d0ececc6a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:37.177991 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7e7cce-d907-4574-8623-69d0ececc6a2 None admin] PATH_INFO: `/v3/projects/6ab53ff2383b487face7ccf9bf1fbecb` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:37.179102 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7e7cce-d907-4574-8623-69d0ececc6a2 None admin] RBAC: Authorizing `identity:get_project(project_id=6ab53ff2383b487face7ccf9bf1fbecb)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:37.180522 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7e7cce-d907-4574-8623-69d0ececc6a2 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:37.182192 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 586/1152] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 24 06:23:37 2022] GET /identity/v3/projects/6ab53ff2383b487face7ccf9bf1fbecb => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:23:37.188386 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7452b08d-90a9-46fe-ae96-b66072b37885 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:37.219958 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:37.294258 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7452b08d-90a9-46fe-ae96-b66072b37885 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:37.327822 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38099d31-a73e-4f34-bd7e-c9b19d8a935a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:37.328487 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38099d31-a73e-4f34-bd7e-c9b19d8a935a None admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:37.328768 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38099d31-a73e-4f34-bd7e-c9b19d8a935a None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:37.329042 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38099d31-a73e-4f34-bd7e-c9b19d8a935a None admin] PATH_INFO: `/v3/projects/6ab53ff2383b487face7ccf9bf1fbecb/users/08da66fcd3e040c581997eab611095bb/roles/486144ad1bcd417e86acdbc1d14327bf` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:37.331522 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38099d31-a73e-4f34-bd7e-c9b19d8a935a None admin] RBAC: Authorizing `identity:create_grant(project_id=6ab53ff2383b487face7ccf9bf1fbecb, user_id=08da66fcd3e040c581997eab611095bb, role_id=486144ad1bcd417e86acdbc1d14327bf)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:37.333071 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38099d31-a73e-4f34-bd7e-c9b19d8a935a None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:37.402191 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 567/1153] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Aug 24 06:23:37 2022] PUT /identity/v3/projects/6ab53ff2383b487face7ccf9bf1fbecb/users/08da66fcd3e040c581997eab611095bb/roles/486144ad1bcd417e86acdbc1d14327bf => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:23:39.292291 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fa16b7-4380-496c-be0b-f11135079362 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:39.292882 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fa16b7-4380-496c-be0b-f11135079362 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:39.292882 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fa16b7-4380-496c-be0b-f11135079362 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:39.294225 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 587/1154] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:23:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:23:39.299767 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c1e10b-4327-4d5d-95d9-5ed9cbd508ca None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:39.300171 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c1e10b-4327-4d5d-95d9-5ed9cbd508ca None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:39.300300 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c1e10b-4327-4d5d-95d9-5ed9cbd508ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:39.318497 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-f1c1e10b-4327-4d5d-95d9-5ed9cbd508ca None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:23:39.319680 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-f1c1e10b-4327-4d5d-95d9-5ed9cbd508ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:23:39.383871 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 568/1155] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:23:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:23:39.652982 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fa16b7-4380-496c-be0b-f11135079362 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:39.753775 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fa16b7-4380-496c-be0b-f11135079362 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:39.788031 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75ea7eb-d096-4aec-a262-a75e9a5b74cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:39.788647 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75ea7eb-d096-4aec-a262-a75e9a5b74cc None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:39.788936 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75ea7eb-d096-4aec-a262-a75e9a5b74cc None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:39.789225 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75ea7eb-d096-4aec-a262-a75e9a5b74cc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:39.799633 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75ea7eb-d096-4aec-a262-a75e9a5b74cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:23:39.804362 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-a75ea7eb-d096-4aec-a262-a75e9a5b74cc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:23:39.804362 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:23:39.804362 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:23:39.804362 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:23:39.804362 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:23:39.804362 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:23:39.804362 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:23:39.804362 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:23:39.804362 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:23:39.804362 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:23:39.804362 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:23:39.804362 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:23:39.804362 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:23:39.804362 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:23:39.804362 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:23:39.804362 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:23:39.805554 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:23:39.805554 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:23:39.805554 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:23:39.805554 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:23:39.805554 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:23:39.805554 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:23:39.805554 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:23:39.805554 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:23:39.805554 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:23:39.805554 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:23:39.805554 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:23:39.805554 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:23:39.805554 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:23:39.805554 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:23:39.805554 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:23:39.805554 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:23:39.806662 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:23:39.806662 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:23:39.806662 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:23:39.806662 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:23:39.806662 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:23:39.806662 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:23:39.806662 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:23:39.806662 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:23:39.806662 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:23:39.806662 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:23:39.806662 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:23:39.806662 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:23:39.806662 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 588/1156] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Aug 24 06:23:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:23:39.811794 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c1e10b-4327-4d5d-95d9-5ed9cbd508ca None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:39.833304 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:39.911816 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c1e10b-4327-4d5d-95d9-5ed9cbd508ca None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:39.945790 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82bb009-645f-4421-8e5b-5791196f3d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:39.946400 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82bb009-645f-4421-8e5b-5791196f3d23 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:39.946679 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82bb009-645f-4421-8e5b-5791196f3d23 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:39.946953 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82bb009-645f-4421-8e5b-5791196f3d23 None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:39.947621 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82bb009-645f-4421-8e5b-5791196f3d23 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:23:39.947924 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82bb009-645f-4421-8e5b-5791196f3d23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:39.949325 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82bb009-645f-4421-8e5b-5791196f3d23 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:39.958733 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 569/1157] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 24 06:23:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:23:39.964733 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75ea7eb-d096-4aec-a262-a75e9a5b74cc None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:39.978476 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:40.068338 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75ea7eb-d096-4aec-a262-a75e9a5b74cc None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:40.102415 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcb6eb9-a364-4665-b026-62f62dac4449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:40.103045 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcb6eb9-a364-4665-b026-62f62dac4449 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:40.103324 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcb6eb9-a364-4665-b026-62f62dac4449 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:40.103597 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcb6eb9-a364-4665-b026-62f62dac4449 None admin] PATH_INFO: `/v3/users/08da66fcd3e040c581997eab611095bb` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:40.104717 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcb6eb9-a364-4665-b026-62f62dac4449 None admin] RBAC: Authorizing `identity:get_user(user_id=08da66fcd3e040c581997eab611095bb)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:40.106161 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcb6eb9-a364-4665-b026-62f62dac4449 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:40.107737 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 589/1158] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 24 06:23:39 2022] GET /identity/v3/users/08da66fcd3e040c581997eab611095bb => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:23:40.114204 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82bb009-645f-4421-8e5b-5791196f3d23 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:40.197549 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:40.220528 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82bb009-645f-4421-8e5b-5791196f3d23 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:40.254686 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6955a8ae-750c-4c89-943a-c7ad53c8a8ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:40.255280 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6955a8ae-750c-4c89-943a-c7ad53c8a8ae None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:40.255555 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6955a8ae-750c-4c89-943a-c7ad53c8a8ae None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:40.255831 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6955a8ae-750c-4c89-943a-c7ad53c8a8ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:40.328676 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6955a8ae-750c-4c89-943a-c7ad53c8a8ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:40.330832 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6955a8ae-750c-4c89-943a-c7ad53c8a8ae None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:40.421962 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:40.442419 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 570/1159] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 24 06:23:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 06:23:40.448478 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcb6eb9-a364-4665-b026-62f62dac4449 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:40.525409 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:40.549051 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcb6eb9-a364-4665-b026-62f62dac4449 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:40.583302 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e0687b-87fb-4e67-a17f-41ed3abaeb41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:40.583925 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e0687b-87fb-4e67-a17f-41ed3abaeb41 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:40.584203 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e0687b-87fb-4e67-a17f-41ed3abaeb41 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:40.584480 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e0687b-87fb-4e67-a17f-41ed3abaeb41 None admin] PATH_INFO: `/v3/projects/6ab53ff2383b487face7ccf9bf1fbecb` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:40.585601 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e0687b-87fb-4e67-a17f-41ed3abaeb41 None admin] RBAC: Authorizing `identity:get_project(project_id=6ab53ff2383b487face7ccf9bf1fbecb)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:40.587013 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e0687b-87fb-4e67-a17f-41ed3abaeb41 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:40.588624 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 590/1160] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 24 06:23:40 2022] GET /identity/v3/projects/6ab53ff2383b487face7ccf9bf1fbecb => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:23:40.594838 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6955a8ae-750c-4c89-943a-c7ad53c8a8ae None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:40.693342 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6955a8ae-750c-4c89-943a-c7ad53c8a8ae None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:40.726620 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d3857a-8d96-4243-bb12-8cf56f23ef59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:40.727154 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d3857a-8d96-4243-bb12-8cf56f23ef59 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:40.727430 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d3857a-8d96-4243-bb12-8cf56f23ef59 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:40.727696 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d3857a-8d96-4243-bb12-8cf56f23ef59 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:40.728410 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d3857a-8d96-4243-bb12-8cf56f23ef59 None admin] RBAC: Adding query filter params (role.id=486144ad1bcd417e86acdbc1d14327bf, scope.project.id=6ab53ff2383b487face7ccf9bf1fbecb, user.id=08da66fcd3e040c581997eab611095bb) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:23:40.728712 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d3857a-8d96-4243-bb12-8cf56f23ef59 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:40.730145 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d3857a-8d96-4243-bb12-8cf56f23ef59 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:40.735888 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 571/1161] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 24 06:23:40 2022] GET /identity/v3/role_assignments?user.id=08da66fcd3e040c581997eab611095bb&scope.project.id=6ab53ff2383b487face7ccf9bf1fbecb&role.id=486144ad1bcd417e86acdbc1d14327bf => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:23:42.650035 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101580ef-5dfc-48b5-9b4e-7be67acda189 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:42.650592 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101580ef-5dfc-48b5-9b4e-7be67acda189 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:42.650592 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101580ef-5dfc-48b5-9b4e-7be67acda189 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:42.651923 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 591/1162] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:23:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 06:23:42.657044 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e67e796-6255-4a38-9918-7707fe146ea6 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:42.657886 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e67e796-6255-4a38-9918-7707fe146ea6 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:42.657886 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e67e796-6255-4a38-9918-7707fe146ea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:42.676380 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-7e67e796-6255-4a38-9918-7707fe146ea6 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:23:42.677619 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-7e67e796-6255-4a38-9918-7707fe146ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:23:42.721087 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:42.744574 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 572/1163] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:23:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:23:43.011198 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101580ef-5dfc-48b5-9b4e-7be67acda189 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:43.118569 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101580ef-5dfc-48b5-9b4e-7be67acda189 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:43.153604 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a63671-2591-46a0-8197-eca2ce213163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:43.154207 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a63671-2591-46a0-8197-eca2ce213163 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:43.154490 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a63671-2591-46a0-8197-eca2ce213163 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:43.154762 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a63671-2591-46a0-8197-eca2ce213163 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:43.168769 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:43.221846 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a63671-2591-46a0-8197-eca2ce213163 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:43.223291 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a63671-2591-46a0-8197-eca2ce213163 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:43.318574 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 592/1164] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 24 06:23:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 06:23:43.324412 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e67e796-6255-4a38-9918-7707fe146ea6 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:43.420835 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e67e796-6255-4a38-9918-7707fe146ea6 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:43.456556 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4f14c3-4e0c-468a-83f4-660710faefb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:43.457227 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4f14c3-4e0c-468a-83f4-660710faefb4 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:43.457505 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4f14c3-4e0c-468a-83f4-660710faefb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:43.457776 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4f14c3-4e0c-468a-83f4-660710faefb4 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:43.465114 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4f14c3-4e0c-468a-83f4-660710faefb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:43.466601 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4f14c3-4e0c-468a-83f4-660710faefb4 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:43.478282 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-2c4f14c3-4e0c-468a-83f4-660710faefb4 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 24 06:23:43.478282 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:23:43.478282 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 24 06:23:43.478282 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 24 06:23:43.478282 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:23:43.478282 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:23:43.478282 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:23:43.478282 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:23:43.478282 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 24 06:23:43.478282 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:23:43.478282 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 06:23:43.478282 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:23:43.478282 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:23:43.478282 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:23:43.478282 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:23:43.478282 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:23:43.478282 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:23:43.479584 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:23:43.479584 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:23:43.479584 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:23:43.479584 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:23:43.479584 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:23:43.479584 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:23:43.479584 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:23:43.479584 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:23:43.479584 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:23:43.479584 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:23:43.479584 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:23:43.479584 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:23:43.479584 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:23:43.479584 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:23:43.479584 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:23:43.479584 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:23:43.480843 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:23:43.480843 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:23:43.480843 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:23:43.480843 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:23:43.480843 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:23:43.480843 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:23:43.480843 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:23:43.480843 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:23:43.480843 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:23:43.480843 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:23:43.480843 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 24 06:23:43.480843 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:23:43.480843 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 24 06:23:43.480843 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:23:43.480843 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 573/1165] 10.222.0.47 () {56 vars in 1117 bytes} [Wed Aug 24 06:23:43 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:23:43.485579 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a63671-2591-46a0-8197-eca2ce213163 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:43.531862 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:43.585939 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a63671-2591-46a0-8197-eca2ce213163 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:43.619074 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5db072-8e2e-4d82-9e76-e225b09073a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:43.619686 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5db072-8e2e-4d82-9e76-e225b09073a5 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:43.619961 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5db072-8e2e-4d82-9e76-e225b09073a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:43.620234 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5db072-8e2e-4d82-9e76-e225b09073a5 None admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:43.620872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5db072-8e2e-4d82-9e76-e225b09073a5 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:23:43.621171 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5db072-8e2e-4d82-9e76-e225b09073a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:43.622566 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5db072-8e2e-4d82-9e76-e225b09073a5 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:43.630525 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 593/1166] 10.222.0.47 () {56 vars in 1115 bytes} [Wed Aug 24 06:23:43 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:23:45.600659 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f70925-f1f8-41fb-aac9-9e69431475b7 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:45.601254 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f70925-f1f8-41fb-aac9-9e69431475b7 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:45.601254 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f70925-f1f8-41fb-aac9-9e69431475b7 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:45.602575 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 574/1167] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:23:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:23:45.607923 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42903b8-5831-44ba-8b8f-10c9738a8100 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:45.608314 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42903b8-5831-44ba-8b8f-10c9738a8100 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:45.608469 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42903b8-5831-44ba-8b8f-10c9738a8100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:45.627532 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-d42903b8-5831-44ba-8b8f-10c9738a8100 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:23:45.628732 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-d42903b8-5831-44ba-8b8f-10c9738a8100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:23:45.697646 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 594/1168] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:23:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:23:45.982892 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f70925-f1f8-41fb-aac9-9e69431475b7 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:46.018489 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:46.085062 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f70925-f1f8-41fb-aac9-9e69431475b7 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:46.119053 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e010fd5-6af3-4ac9-9217-afae23f58953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:46.119656 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e010fd5-6af3-4ac9-9217-afae23f58953 None admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:46.119934 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e010fd5-6af3-4ac9-9217-afae23f58953 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:46.120198 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e010fd5-6af3-4ac9-9217-afae23f58953 None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:46.120856 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e010fd5-6af3-4ac9-9217-afae23f58953 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:46.122259 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e010fd5-6af3-4ac9-9217-afae23f58953 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:46.185276 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 575/1169] 10.222.0.47 () {60 vars in 1138 bytes} [Wed Aug 24 06:23:45 2022] POST /identity/v3/domains => generated 267 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:23:48.125751 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a60f3d-66e0-4436-aa8b-1ec54c54e0a9 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:48.126648 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a60f3d-66e0-4436-aa8b-1ec54c54e0a9 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:48.126648 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a60f3d-66e0-4436-aa8b-1ec54c54e0a9 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:48.128227 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 595/1170] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:23:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 06:23:48.135514 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7420be1e-30f3-48bf-90bd-47a2346ee36b None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:48.136066 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7420be1e-30f3-48bf-90bd-47a2346ee36b None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:48.136478 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7420be1e-30f3-48bf-90bd-47a2346ee36b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:48.155610 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-7420be1e-30f3-48bf-90bd-47a2346ee36b None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:23:48.156826 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-7420be1e-30f3-48bf-90bd-47a2346ee36b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:23:48.234388 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 576/1171] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:23:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:23:48.520668 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a60f3d-66e0-4436-aa8b-1ec54c54e0a9 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:48.563199 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:48.624688 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a60f3d-66e0-4436-aa8b-1ec54c54e0a9 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:48.660799 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c4f04b-7044-4b2e-999e-18e19a86d59c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:48.661471 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c4f04b-7044-4b2e-999e-18e19a86d59c None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:48.661754 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c4f04b-7044-4b2e-999e-18e19a86d59c None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:48.662031 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c4f04b-7044-4b2e-999e-18e19a86d59c None admin] PATH_INFO: `/v3/domains/217ea54aaccc4b00a67c97858033aab1` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:48.672657 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c4f04b-7044-4b2e-999e-18e19a86d59c None admin] RBAC: Authorizing `identity:get_domain(domain_id=217ea54aaccc4b00a67c97858033aab1)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:48.674208 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c4f04b-7044-4b2e-999e-18e19a86d59c None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:48.675856 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 596/1172] 10.222.0.47 () {56 vars in 1183 bytes} [Wed Aug 24 06:23:48 2022] GET /identity/v3/domains/217ea54aaccc4b00a67c97858033aab1 => generated 267 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:23:48.681742 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7420be1e-30f3-48bf-90bd-47a2346ee36b None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:48.708465 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:48.785272 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7420be1e-30f3-48bf-90bd-47a2346ee36b None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:48.820270 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8417338-2f5c-4945-9a70-8803b9745580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:48.820883 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8417338-2f5c-4945-9a70-8803b9745580 None admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:48.821156 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8417338-2f5c-4945-9a70-8803b9745580 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:48.821452 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8417338-2f5c-4945-9a70-8803b9745580 None admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:48.822094 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8417338-2f5c-4945-9a70-8803b9745580 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:48.823518 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8417338-2f5c-4945-9a70-8803b9745580 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:48.974514 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 577/1173] 10.222.0.47 () {60 vars in 1141 bytes} [Wed Aug 24 06:23:48 2022] POST /identity/v3/projects => generated 361 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:23:50.936807 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702792e2-2ca6-4380-9b85-e824a75d8c80 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:50.937359 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702792e2-2ca6-4380-9b85-e824a75d8c80 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:50.937359 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702792e2-2ca6-4380-9b85-e824a75d8c80 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:50.938641 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 597/1174] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:23:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:23:50.944083 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b37d78f-15f5-4540-a89b-d4bd6baa49c8 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:50.944537 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b37d78f-15f5-4540-a89b-d4bd6baa49c8 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:50.944649 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b37d78f-15f5-4540-a89b-d4bd6baa49c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:50.963394 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-8b37d78f-15f5-4540-a89b-d4bd6baa49c8 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:23:50.964594 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-8b37d78f-15f5-4540-a89b-d4bd6baa49c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:23:51.033501 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 578/1175] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:23:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:23:51.302022 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702792e2-2ca6-4380-9b85-e824a75d8c80 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:51.399346 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:51.403828 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702792e2-2ca6-4380-9b85-e824a75d8c80 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:51.438708 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3879da37-284b-44c4-9762-792593702089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:51.439318 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3879da37-284b-44c4-9762-792593702089 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:51.439593 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3879da37-284b-44c4-9762-792593702089 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:51.439859 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3879da37-284b-44c4-9762-792593702089 None admin] PATH_INFO: `/v3/domains/217ea54aaccc4b00a67c97858033aab1` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:51.440961 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3879da37-284b-44c4-9762-792593702089 None admin] RBAC: Authorizing `identity:get_domain(domain_id=217ea54aaccc4b00a67c97858033aab1)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:51.442406 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3879da37-284b-44c4-9762-792593702089 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:51.444360 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 598/1176] 10.222.0.47 () {56 vars in 1183 bytes} [Wed Aug 24 06:23:51 2022] GET /identity/v3/domains/217ea54aaccc4b00a67c97858033aab1 => generated 267 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:23:51.449778 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b37d78f-15f5-4540-a89b-d4bd6baa49c8 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:51.459668 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:51.545762 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b37d78f-15f5-4540-a89b-d4bd6baa49c8 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:51.578450 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e6b845-7fd3-4fd1-b8ae-ba270a70a7a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:51.579496 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e6b845-7fd3-4fd1-b8ae-ba270a70a7a2 None admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:51.579793 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e6b845-7fd3-4fd1-b8ae-ba270a70a7a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:51.580044 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e6b845-7fd3-4fd1-b8ae-ba270a70a7a2 None admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:51.580683 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e6b845-7fd3-4fd1-b8ae-ba270a70a7a2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:51.582134 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e6b845-7fd3-4fd1-b8ae-ba270a70a7a2 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:51.695628 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 579/1177] 10.222.0.47 () {60 vars in 1132 bytes} [Wed Aug 24 06:23:51 2022] POST /identity/v3/users => generated 311 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:23:53.664998 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a604a1ce-6dee-4720-ba46-ea22b7774ba3 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:53.665568 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a604a1ce-6dee-4720-ba46-ea22b7774ba3 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:53.665568 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a604a1ce-6dee-4720-ba46-ea22b7774ba3 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:53.666935 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 599/1178] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:23:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:23:53.672275 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a62867-4837-47c2-acb5-36186a068906 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:53.673383 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a62867-4837-47c2-acb5-36186a068906 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:53.673383 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a62867-4837-47c2-acb5-36186a068906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:53.691640 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-25a62867-4837-47c2-acb5-36186a068906 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:23:53.692829 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-25a62867-4837-47c2-acb5-36186a068906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:23:53.752405 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:53.760755 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 580/1179] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:23:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:23:54.036794 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a604a1ce-6dee-4720-ba46-ea22b7774ba3 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:54.135803 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a604a1ce-6dee-4720-ba46-ea22b7774ba3 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:54.168336 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d9870c-e3f6-414d-b8d3-a3e89311c203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:54.168947 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d9870c-e3f6-414d-b8d3-a3e89311c203 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:54.169256 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d9870c-e3f6-414d-b8d3-a3e89311c203 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:54.169529 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d9870c-e3f6-414d-b8d3-a3e89311c203 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:54.179771 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d9870c-e3f6-414d-b8d3-a3e89311c203 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:23:54.184389 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-d6d9870c-e3f6-414d-b8d3-a3e89311c203 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:23:54.184389 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:23:54.184389 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:23:54.184389 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:23:54.184389 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:23:54.184389 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:23:54.184389 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:23:54.184389 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:23:54.184389 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:23:54.184389 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:23:54.184389 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:23:54.184389 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:23:54.184389 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:23:54.184389 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:23:54.184389 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:23:54.184389 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:23:54.185542 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:23:54.185542 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:23:54.185542 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:23:54.185542 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:23:54.185542 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:23:54.185542 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:23:54.185542 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:23:54.185542 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:23:54.185542 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:23:54.185542 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:23:54.185542 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:23:54.185542 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:23:54.185542 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:23:54.185542 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:23:54.185542 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:23:54.185542 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:23:54.186400 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:23:54.186400 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:23:54.186400 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:23:54.186400 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:23:54.186400 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:23:54.186400 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:23:54.186400 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:23:54.186400 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:23:54.186400 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:23:54.186400 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:23:54.186400 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:23:54.186400 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:23:54.186400 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 600/1180] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Aug 24 06:23:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:23:54.192477 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a62867-4837-47c2-acb5-36186a068906 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:54.289958 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a62867-4837-47c2-acb5-36186a068906 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:54.323025 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74decd8e-660e-4d68-832d-38a8d736e9c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:54.323632 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74decd8e-660e-4d68-832d-38a8d736e9c4 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:54.323915 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74decd8e-660e-4d68-832d-38a8d736e9c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:54.324191 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74decd8e-660e-4d68-832d-38a8d736e9c4 None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:54.324856 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74decd8e-660e-4d68-832d-38a8d736e9c4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:23:54.325163 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74decd8e-660e-4d68-832d-38a8d736e9c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:54.326555 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74decd8e-660e-4d68-832d-38a8d736e9c4 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:54.334781 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 581/1181] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 24 06:23:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:23:54.340653 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d9870c-e3f6-414d-b8d3-a3e89311c203 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:54.369550 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:54.442248 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d9870c-e3f6-414d-b8d3-a3e89311c203 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:54.476080 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ed3248-8932-488c-8004-958124f6adab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:54.476701 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ed3248-8932-488c-8004-958124f6adab None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:54.476981 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ed3248-8932-488c-8004-958124f6adab None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:54.477288 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ed3248-8932-488c-8004-958124f6adab None admin] PATH_INFO: `/v3/users/8cc41b4e2dd54aba894a3c3ce33394e7` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:54.490852 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ed3248-8932-488c-8004-958124f6adab None admin] RBAC: Authorizing `identity:get_user(user_id=8cc41b4e2dd54aba894a3c3ce33394e7)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:54.492293 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ed3248-8932-488c-8004-958124f6adab None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:54.493927 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 601/1182] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 24 06:23:54 2022] GET /identity/v3/users/8cc41b4e2dd54aba894a3c3ce33394e7 => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:23:54.500049 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74decd8e-660e-4d68-832d-38a8d736e9c4 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:54.557263 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:54.601866 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74decd8e-660e-4d68-832d-38a8d736e9c4 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:54.635091 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbe032b-4fa8-45c0-b6a0-2dbf103ba3c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:54.635698 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbe032b-4fa8-45c0-b6a0-2dbf103ba3c7 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:54.635978 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbe032b-4fa8-45c0-b6a0-2dbf103ba3c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:54.636258 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbe032b-4fa8-45c0-b6a0-2dbf103ba3c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:54.705318 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbe032b-4fa8-45c0-b6a0-2dbf103ba3c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:54.706772 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbe032b-4fa8-45c0-b6a0-2dbf103ba3c7 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:54.761772 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:54.806762 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 582/1183] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 24 06:23:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 06:23:54.812684 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ed3248-8932-488c-8004-958124f6adab None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:54.895931 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:54.911618 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ed3248-8932-488c-8004-958124f6adab None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:54.946477 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0491e1fe-28be-4cba-8815-9ddbf350372b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:54.947111 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0491e1fe-28be-4cba-8815-9ddbf350372b None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:54.947392 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0491e1fe-28be-4cba-8815-9ddbf350372b None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:54.947669 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0491e1fe-28be-4cba-8815-9ddbf350372b None admin] PATH_INFO: `/v3/projects/03ea0a8f82fa4361b70adcb83f24d4de` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:54.948792 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0491e1fe-28be-4cba-8815-9ddbf350372b None admin] RBAC: Authorizing `identity:get_project(project_id=03ea0a8f82fa4361b70adcb83f24d4de)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:54.950255 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0491e1fe-28be-4cba-8815-9ddbf350372b None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:54.951899 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 602/1184] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 24 06:23:54 2022] GET /identity/v3/projects/03ea0a8f82fa4361b70adcb83f24d4de => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:23:54.959224 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbe032b-4fa8-45c0-b6a0-2dbf103ba3c7 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:55.061593 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbe032b-4fa8-45c0-b6a0-2dbf103ba3c7 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:55.096051 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613b7ad8-7379-4ee7-a533-c7459c77258a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:55.096683 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613b7ad8-7379-4ee7-a533-c7459c77258a None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:55.097020 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613b7ad8-7379-4ee7-a533-c7459c77258a None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:55.097123 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613b7ad8-7379-4ee7-a533-c7459c77258a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:55.097856 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613b7ad8-7379-4ee7-a533-c7459c77258a None admin] RBAC: Adding query filter params (role.id=486144ad1bcd417e86acdbc1d14327bf, scope.project.id=03ea0a8f82fa4361b70adcb83f24d4de, user.id=8cc41b4e2dd54aba894a3c3ce33394e7) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:23:55.098258 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613b7ad8-7379-4ee7-a533-c7459c77258a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:55.099569 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613b7ad8-7379-4ee7-a533-c7459c77258a None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:55.105985 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 583/1185] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 24 06:23:54 2022] GET /identity/v3/role_assignments?user.id=8cc41b4e2dd54aba894a3c3ce33394e7&scope.project.id=03ea0a8f82fa4361b70adcb83f24d4de&role.id=486144ad1bcd417e86acdbc1d14327bf => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:23:57.051071 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d69314-f49d-4cd0-b421-0b1dc96ff50f None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:57.051623 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d69314-f49d-4cd0-b421-0b1dc96ff50f None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:57.051623 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d69314-f49d-4cd0-b421-0b1dc96ff50f None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:57.052854 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 603/1186] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:23:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:23:57.058152 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7698593-1d12-42f0-8ae2-f385791444df None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:57.058548 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7698593-1d12-42f0-8ae2-f385791444df None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:57.058689 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7698593-1d12-42f0-8ae2-f385791444df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:57.076326 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-f7698593-1d12-42f0-8ae2-f385791444df None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:23:57.077811 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-f7698593-1d12-42f0-8ae2-f385791444df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:23:57.093422 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:57.151723 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 584/1187] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:23:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 06:23:57.427795 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d69314-f49d-4cd0-b421-0b1dc96ff50f None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:57.530644 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d69314-f49d-4cd0-b421-0b1dc96ff50f None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:57.565030 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fd63da-ba5e-46de-b7b7-2175031dbb29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:57.565688 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fd63da-ba5e-46de-b7b7-2175031dbb29 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:57.565968 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fd63da-ba5e-46de-b7b7-2175031dbb29 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:57.566239 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fd63da-ba5e-46de-b7b7-2175031dbb29 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:57.576023 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fd63da-ba5e-46de-b7b7-2175031dbb29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:23:57.581562 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-c6fd63da-ba5e-46de-b7b7-2175031dbb29 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:23:57.581562 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:23:57.581562 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:23:57.581562 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:23:57.581562 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:23:57.581562 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:23:57.581562 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:23:57.581562 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:23:57.581562 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:23:57.581562 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:23:57.581562 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:23:57.581562 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:23:57.581562 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:23:57.581562 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:23:57.581562 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:23:57.581562 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:23:57.582670 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:23:57.582670 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:23:57.582670 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:23:57.582670 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:23:57.582670 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:23:57.582670 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:23:57.582670 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:23:57.582670 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:23:57.582670 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:23:57.582670 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:23:57.582670 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:23:57.582670 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:23:57.582670 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:23:57.582670 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:23:57.582670 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:23:57.582670 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:23:57.583544 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:23:57.583544 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:23:57.583544 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:23:57.583544 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:23:57.583544 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:23:57.583544 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:23:57.583544 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:23:57.583544 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:23:57.583544 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:23:57.583544 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:23:57.583544 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:23:57.583544 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:23:57.583544 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 604/1188] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Aug 24 06:23:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:23:57.588854 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7698593-1d12-42f0-8ae2-f385791444df None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:57.687228 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7698593-1d12-42f0-8ae2-f385791444df None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:57.721749 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fbd673-1ce1-4110-af2a-19c0df1243d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:57.722362 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fbd673-1ce1-4110-af2a-19c0df1243d5 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:57.722642 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fbd673-1ce1-4110-af2a-19c0df1243d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:57.722914 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fbd673-1ce1-4110-af2a-19c0df1243d5 None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:57.723581 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fbd673-1ce1-4110-af2a-19c0df1243d5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:23:57.723885 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fbd673-1ce1-4110-af2a-19c0df1243d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:57.725262 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fbd673-1ce1-4110-af2a-19c0df1243d5 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:57.733347 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 585/1189] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 24 06:23:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:23:57.740494 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fd63da-ba5e-46de-b7b7-2175031dbb29 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:57.757383 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:57.840932 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fd63da-ba5e-46de-b7b7-2175031dbb29 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:57.875078 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78151004-f222-4322-9c0a-b30792d4fc93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:57.876260 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78151004-f222-4322-9c0a-b30792d4fc93 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:57.876533 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78151004-f222-4322-9c0a-b30792d4fc93 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:57.876807 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78151004-f222-4322-9c0a-b30792d4fc93 None admin] PATH_INFO: `/v3/users/8cc41b4e2dd54aba894a3c3ce33394e7` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:57.877968 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78151004-f222-4322-9c0a-b30792d4fc93 None admin] RBAC: Authorizing `identity:get_user(user_id=8cc41b4e2dd54aba894a3c3ce33394e7)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:57.879386 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78151004-f222-4322-9c0a-b30792d4fc93 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:57.880960 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 605/1190] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 24 06:23:57 2022] GET /identity/v3/users/8cc41b4e2dd54aba894a3c3ce33394e7 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:23:57.886970 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fbd673-1ce1-4110-af2a-19c0df1243d5 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:57.902616 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:57.988678 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fbd673-1ce1-4110-af2a-19c0df1243d5 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:58.021294 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d20a5a9-5d29-4b35-afc9-5e16fba2ee10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:58.021935 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d20a5a9-5d29-4b35-afc9-5e16fba2ee10 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:58.022218 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d20a5a9-5d29-4b35-afc9-5e16fba2ee10 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:58.022491 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d20a5a9-5d29-4b35-afc9-5e16fba2ee10 None admin] PATH_INFO: `/v3/projects/03ea0a8f82fa4361b70adcb83f24d4de` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:58.023613 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d20a5a9-5d29-4b35-afc9-5e16fba2ee10 None admin] RBAC: Authorizing `identity:get_project(project_id=03ea0a8f82fa4361b70adcb83f24d4de)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:58.025034 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d20a5a9-5d29-4b35-afc9-5e16fba2ee10 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:58.026693 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 586/1191] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 24 06:23:57 2022] GET /identity/v3/projects/03ea0a8f82fa4361b70adcb83f24d4de => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:23:58.033456 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78151004-f222-4322-9c0a-b30792d4fc93 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:23:58.122032 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:23:58.140406 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78151004-f222-4322-9c0a-b30792d4fc93 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:23:58.173691 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264655e5-cbb6-4db0-8e10-ac3d5c4e52b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:23:58.174336 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264655e5-cbb6-4db0-8e10-ac3d5c4e52b3 None admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:23:58.174621 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264655e5-cbb6-4db0-8e10-ac3d5c4e52b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:23:58.174918 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264655e5-cbb6-4db0-8e10-ac3d5c4e52b3 None admin] PATH_INFO: `/v3/projects/03ea0a8f82fa4361b70adcb83f24d4de/users/8cc41b4e2dd54aba894a3c3ce33394e7/roles/486144ad1bcd417e86acdbc1d14327bf` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:23:58.176940 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264655e5-cbb6-4db0-8e10-ac3d5c4e52b3 None admin] RBAC: Authorizing `identity:create_grant(project_id=03ea0a8f82fa4361b70adcb83f24d4de, user_id=8cc41b4e2dd54aba894a3c3ce33394e7, role_id=486144ad1bcd417e86acdbc1d14327bf)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:23:58.178764 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264655e5-cbb6-4db0-8e10-ac3d5c4e52b3 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:23:58.243344 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 606/1192] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Aug 24 06:23:58 2022] PUT /identity/v3/projects/03ea0a8f82fa4361b70adcb83f24d4de/users/8cc41b4e2dd54aba894a3c3ce33394e7/roles/486144ad1bcd417e86acdbc1d14327bf => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:24:00.145019 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb68307f-e41f-4a01-aa2a-b810642c65d4 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:24:00.145617 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb68307f-e41f-4a01-aa2a-b810642c65d4 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:24:00.145617 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb68307f-e41f-4a01-aa2a-b810642c65d4 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:24:00.146942 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 587/1193] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:24:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:24:00.152205 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51bf0cf-c355-4579-8919-6df1e382a1ad None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:24:00.152590 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51bf0cf-c355-4579-8919-6df1e382a1ad None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:24:00.152763 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51bf0cf-c355-4579-8919-6df1e382a1ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:24:00.170992 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-c51bf0cf-c355-4579-8919-6df1e382a1ad None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:24:00.172721 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-c51bf0cf-c355-4579-8919-6df1e382a1ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:24:00.239172 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 607/1194] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:24:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:24:00.506142 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb68307f-e41f-4a01-aa2a-b810642c65d4 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:24:00.588279 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:24:00.610111 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb68307f-e41f-4a01-aa2a-b810642c65d4 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:24:00.645222 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f0cd15-8629-42e5-802e-6009bde5df23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:24:00.645856 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f0cd15-8629-42e5-802e-6009bde5df23 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:24:00.646151 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f0cd15-8629-42e5-802e-6009bde5df23 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:24:00.646423 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f0cd15-8629-42e5-802e-6009bde5df23 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:24:00.657625 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f0cd15-8629-42e5-802e-6009bde5df23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:24:00.662286 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-a0f0cd15-8629-42e5-802e-6009bde5df23 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:24:00.662286 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:24:00.662286 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:24:00.662286 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:24:00.662286 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:24:00.662286 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:24:00.662286 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:24:00.662286 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:24:00.662286 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:24:00.662286 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:24:00.662286 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:24:00.662286 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:24:00.662286 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:24:00.662286 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:24:00.662286 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:24:00.662286 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:24:00.663376 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:24:00.663376 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:24:00.663376 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:24:00.663376 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:24:00.663376 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:24:00.663376 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:24:00.663376 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:24:00.663376 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:24:00.663376 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:24:00.663376 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:24:00.663376 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:24:00.663376 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:24:00.663376 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:24:00.663376 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:24:00.663376 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:24:00.663376 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:24:00.664403 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:24:00.664403 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:24:00.664403 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:24:00.664403 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:24:00.664403 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:24:00.664403 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:24:00.664403 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:24:00.664403 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:24:00.664403 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:24:00.664403 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:24:00.664403 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:24:00.664403 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:24:00.664403 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 588/1195] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Aug 24 06:24:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:24:00.669541 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51bf0cf-c355-4579-8919-6df1e382a1ad None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:24:00.738300 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:24:00.768772 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51bf0cf-c355-4579-8919-6df1e382a1ad None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:24:00.803280 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ba4ba0-c35b-4c48-ba4d-bdaf9b524849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:24:00.803880 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ba4ba0-c35b-4c48-ba4d-bdaf9b524849 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:24:00.804162 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ba4ba0-c35b-4c48-ba4d-bdaf9b524849 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:24:00.804436 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ba4ba0-c35b-4c48-ba4d-bdaf9b524849 None admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:24:00.805096 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ba4ba0-c35b-4c48-ba4d-bdaf9b524849 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:24:00.805427 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ba4ba0-c35b-4c48-ba4d-bdaf9b524849 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:24:00.806794 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ba4ba0-c35b-4c48-ba4d-bdaf9b524849 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:24:00.815313 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 608/1196] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 24 06:24:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:24:00.821153 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f0cd15-8629-42e5-802e-6009bde5df23 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:24:00.929707 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f0cd15-8629-42e5-802e-6009bde5df23 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:24:00.969923 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baf785d-4251-4755-b42c-67720bb47747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:24:00.970548 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baf785d-4251-4755-b42c-67720bb47747 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:24:00.970827 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baf785d-4251-4755-b42c-67720bb47747 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:24:00.971097 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baf785d-4251-4755-b42c-67720bb47747 None admin] PATH_INFO: `/v3/users/8cc41b4e2dd54aba894a3c3ce33394e7` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:24:00.972231 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baf785d-4251-4755-b42c-67720bb47747 None admin] RBAC: Authorizing `identity:get_user(user_id=8cc41b4e2dd54aba894a3c3ce33394e7)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:24:00.973674 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baf785d-4251-4755-b42c-67720bb47747 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:24:00.975249 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 589/1197] 10.222.0.47 () {56 vars in 1177 bytes} [Wed Aug 24 06:24:00 2022] GET /identity/v3/users/8cc41b4e2dd54aba894a3c3ce33394e7 => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:24:00.981334 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ba4ba0-c35b-4c48-ba4d-bdaf9b524849 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:24:01.082014 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ba4ba0-c35b-4c48-ba4d-bdaf9b524849 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:24:01.111009 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:24:01.115870 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da87a58-eab4-4e53-a310-4b2ee915cad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:24:01.116483 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da87a58-eab4-4e53-a310-4b2ee915cad4 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:24:01.121769 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da87a58-eab4-4e53-a310-4b2ee915cad4 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:24:01.121769 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da87a58-eab4-4e53-a310-4b2ee915cad4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:24:01.188717 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da87a58-eab4-4e53-a310-4b2ee915cad4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:24:01.190211 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da87a58-eab4-4e53-a310-4b2ee915cad4 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:24:01.287863 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 609/1198] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 24 06:24:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 06:24:01.293899 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baf785d-4251-4755-b42c-67720bb47747 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:24:01.306298 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:24:01.394707 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baf785d-4251-4755-b42c-67720bb47747 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:24:01.428595 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c367d2d8-8a29-4967-8697-cf040ae7d048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:24:01.429271 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c367d2d8-8a29-4967-8697-cf040ae7d048 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:24:01.429558 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c367d2d8-8a29-4967-8697-cf040ae7d048 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:24:01.429835 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c367d2d8-8a29-4967-8697-cf040ae7d048 None admin] PATH_INFO: `/v3/projects/03ea0a8f82fa4361b70adcb83f24d4de` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:24:01.430957 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c367d2d8-8a29-4967-8697-cf040ae7d048 None admin] RBAC: Authorizing `identity:get_project(project_id=03ea0a8f82fa4361b70adcb83f24d4de)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:24:01.432379 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c367d2d8-8a29-4967-8697-cf040ae7d048 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:24:01.434048 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 590/1199] 10.222.0.47 () {56 vars in 1186 bytes} [Wed Aug 24 06:24:01 2022] GET /identity/v3/projects/03ea0a8f82fa4361b70adcb83f24d4de => generated 361 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:24:01.441536 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da87a58-eab4-4e53-a310-4b2ee915cad4 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:24:01.471656 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:24:01.543410 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da87a58-eab4-4e53-a310-4b2ee915cad4 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:24:01.577308 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae6bfe4-705f-4e1f-8220-37f4e5f09575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:24:01.577905 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae6bfe4-705f-4e1f-8220-37f4e5f09575 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:24:01.578310 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae6bfe4-705f-4e1f-8220-37f4e5f09575 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:24:01.578615 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae6bfe4-705f-4e1f-8220-37f4e5f09575 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:24:01.579376 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae6bfe4-705f-4e1f-8220-37f4e5f09575 None admin] RBAC: Adding query filter params (role.id=486144ad1bcd417e86acdbc1d14327bf, scope.project.id=03ea0a8f82fa4361b70adcb83f24d4de, user.id=8cc41b4e2dd54aba894a3c3ce33394e7) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:24:01.579681 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae6bfe4-705f-4e1f-8220-37f4e5f09575 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:24:01.581105 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae6bfe4-705f-4e1f-8220-37f4e5f09575 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:24:01.586769 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 610/1200] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 24 06:24:01 2022] GET /identity/v3/role_assignments?user.id=8cc41b4e2dd54aba894a3c3ce33394e7&scope.project.id=03ea0a8f82fa4361b70adcb83f24d4de&role.id=486144ad1bcd417e86acdbc1d14327bf => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:24:11.424240 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=62 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:24:11.584204 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=52 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:25:16.481335 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44042c02-bc8f-423c-90ad-65515a6df399 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:16.481915 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44042c02-bc8f-423c-90ad-65515a6df399 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:16.481915 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44042c02-bc8f-423c-90ad-65515a6df399 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:16.483215 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 591/1201] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:25:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:25:16.488328 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0855f5a-ec64-4ac2-952e-f09d1cd2ebca None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:16.488816 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0855f5a-ec64-4ac2-952e-f09d1cd2ebca None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:16.488914 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0855f5a-ec64-4ac2-952e-f09d1cd2ebca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:16.516740 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-e0855f5a-ec64-4ac2-952e-f09d1cd2ebca None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:25:16.519829 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-e0855f5a-ec64-4ac2-952e-f09d1cd2ebca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:25:16.603922 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 611/1202] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:25:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:25:16.882005 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44042c02-bc8f-423c-90ad-65515a6df399 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:16.991291 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44042c02-bc8f-423c-90ad-65515a6df399 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:17.025141 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b424dd90-cc12-4963-9dd3-3b78b8f12e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:17.025789 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b424dd90-cc12-4963-9dd3-3b78b8f12e70 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:17.026064 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b424dd90-cc12-4963-9dd3-3b78b8f12e70 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:17.026333 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b424dd90-cc12-4963-9dd3-3b78b8f12e70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:17.036998 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b424dd90-cc12-4963-9dd3-3b78b8f12e70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:25:17.039252 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b424dd90-cc12-4963-9dd3-3b78b8f12e70 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:25:17.052819 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-b424dd90-cc12-4963-9dd3-3b78b8f12e70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:25:17.052819 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:25:17.052819 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:25:17.052819 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:25:17.052819 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:25:17.052819 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:25:17.052819 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:25:17.052819 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:25:17.052819 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:25:17.052819 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:25:17.052819 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:25:17.052819 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:25:17.052819 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:25:17.052819 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:25:17.052819 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:25:17.052819 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:25:17.054143 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:25:17.054143 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:25:17.054143 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:25:17.054143 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:25:17.054143 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:25:17.054143 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:25:17.054143 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:25:17.054143 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:25:17.054143 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:25:17.054143 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:25:17.054143 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:25:17.054143 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:25:17.054143 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:25:17.054143 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:25:17.054143 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:25:17.054143 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:25:17.054143 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:25:17.055398 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:25:17.055398 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:25:17.055398 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:25:17.055398 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 592/1203] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:25:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:25:17.060309 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0855f5a-ec64-4ac2-952e-f09d1cd2ebca None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:17.160021 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0855f5a-ec64-4ac2-952e-f09d1cd2ebca None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:17.188001 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:25:17.194302 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de62423-5321-4455-a790-ed22c0c93790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:17.197796 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de62423-5321-4455-a790-ed22c0c93790 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:17.197796 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de62423-5321-4455-a790-ed22c0c93790 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:17.197796 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de62423-5321-4455-a790-ed22c0c93790 None admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:17.197796 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de62423-5321-4455-a790-ed22c0c93790 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:25:17.197796 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de62423-5321-4455-a790-ed22c0c93790 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:25:17.197796 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de62423-5321-4455-a790-ed22c0c93790 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:25:17.209819 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 612/1204] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 24 06:25:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:25:17.215856 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b424dd90-cc12-4963-9dd3-3b78b8f12e70 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:17.267991 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:25:17.314057 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b424dd90-cc12-4963-9dd3-3b78b8f12e70 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:17.347177 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2324086b-2b4c-4d53-8b76-fa4d29da2cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:17.347861 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2324086b-2b4c-4d53-8b76-fa4d29da2cca None admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:17.348140 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2324086b-2b4c-4d53-8b76-fa4d29da2cca None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:17.348410 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2324086b-2b4c-4d53-8b76-fa4d29da2cca None admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:17.349051 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2324086b-2b4c-4d53-8b76-fa4d29da2cca None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:25:17.350514 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2324086b-2b4c-4d53-8b76-fa4d29da2cca None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:25:17.453797 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 593/1205] 10.222.0.47 () {60 vars in 1132 bytes} [Wed Aug 24 06:25:17 2022] POST /identity/v3/users => generated 251 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:25:19.413437 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eb7803-cca8-4507-b509-f2c182204f73 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:19.414003 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eb7803-cca8-4507-b509-f2c182204f73 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:19.414003 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eb7803-cca8-4507-b509-f2c182204f73 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:19.415336 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 613/1206] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:25:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:25:19.420598 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d1e4b6-85f2-4291-a37e-9cd6837872a4 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:19.421079 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d1e4b6-85f2-4291-a37e-9cd6837872a4 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:19.421221 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d1e4b6-85f2-4291-a37e-9cd6837872a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:19.439543 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-e3d1e4b6-85f2-4291-a37e-9cd6837872a4 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:25:19.441369 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-e3d1e4b6-85f2-4291-a37e-9cd6837872a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:25:19.507863 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 594/1207] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:25:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:25:19.778024 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eb7803-cca8-4507-b509-f2c182204f73 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:19.882345 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eb7803-cca8-4507-b509-f2c182204f73 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:19.916732 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a46ab0-8c74-4102-9e7b-5cf0793d50e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:19.917450 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a46ab0-8c74-4102-9e7b-5cf0793d50e6 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:19.917700 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a46ab0-8c74-4102-9e7b-5cf0793d50e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:19.917976 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a46ab0-8c74-4102-9e7b-5cf0793d50e6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:19.928589 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a46ab0-8c74-4102-9e7b-5cf0793d50e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:25:19.933326 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-77a46ab0-8c74-4102-9e7b-5cf0793d50e6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:25:19.933326 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:25:19.933326 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:25:19.933326 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:25:19.933326 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:25:19.933326 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:25:19.933326 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:25:19.933326 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:25:19.933326 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:25:19.933326 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:25:19.933326 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:25:19.933326 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:25:19.933326 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:25:19.933326 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:25:19.933326 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:25:19.933326 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:25:19.934493 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:25:19.934493 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:25:19.934493 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:25:19.934493 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:25:19.934493 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:25:19.934493 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:25:19.934493 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:25:19.934493 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:25:19.934493 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:25:19.934493 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:25:19.934493 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:25:19.934493 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:25:19.934493 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:25:19.934493 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:25:19.934493 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:25:19.934493 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:25:19.935399 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:25:19.935399 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:25:19.935399 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:25:19.935399 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:25:19.935399 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:25:19.935399 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:25:19.935399 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:25:19.935399 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:25:19.935399 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:25:19.935399 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:25:19.935399 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:25:19.935399 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:25:19.935399 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 614/1208] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Aug 24 06:25:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:25:19.940666 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d1e4b6-85f2-4291-a37e-9cd6837872a4 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:19.981067 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:25:20.039199 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d1e4b6-85f2-4291-a37e-9cd6837872a4 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:20.073255 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7924e0d1-1985-4f6f-8ff6-6b27d641ed54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:20.073861 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7924e0d1-1985-4f6f-8ff6-6b27d641ed54 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:20.074141 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7924e0d1-1985-4f6f-8ff6-6b27d641ed54 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:20.074413 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7924e0d1-1985-4f6f-8ff6-6b27d641ed54 None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:20.075075 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7924e0d1-1985-4f6f-8ff6-6b27d641ed54 None admin] RBAC: Adding query filter params (name=service) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:25:20.075380 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7924e0d1-1985-4f6f-8ff6-6b27d641ed54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:25:20.076749 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7924e0d1-1985-4f6f-8ff6-6b27d641ed54 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:25:20.085016 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 595/1209] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Aug 24 06:25:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:25:20.090870 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a46ab0-8c74-4102-9e7b-5cf0793d50e6 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:20.148152 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:25:20.197560 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a46ab0-8c74-4102-9e7b-5cf0793d50e6 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:20.231406 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0618d06-de2e-4665-9886-0110a7242a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:20.232034 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0618d06-de2e-4665-9886-0110a7242a31 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:20.232318 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0618d06-de2e-4665-9886-0110a7242a31 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:20.232592 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0618d06-de2e-4665-9886-0110a7242a31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:20.242961 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0618d06-de2e-4665-9886-0110a7242a31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:25:20.247606 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0618d06-de2e-4665-9886-0110a7242a31 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:25:20.259461 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-a0618d06-de2e-4665-9886-0110a7242a31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:25:20.259461 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:25:20.259461 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:25:20.259461 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:25:20.259461 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:25:20.259461 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:25:20.259461 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:25:20.259461 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:25:20.259461 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:25:20.259461 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:25:20.259461 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:25:20.259461 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:25:20.259461 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:25:20.259461 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:25:20.259461 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:25:20.259461 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:25:20.260567 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:25:20.260567 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:25:20.260567 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:25:20.260567 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:25:20.260567 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:25:20.260567 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:25:20.260567 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:25:20.260567 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:25:20.260567 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:25:20.260567 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:25:20.260567 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:25:20.260567 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:25:20.260567 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:25:20.260567 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:25:20.260567 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:25:20.260567 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:25:20.260567 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:25:20.261538 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:25:20.261538 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:25:20.261538 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:25:20.261538 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 615/1210] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:25:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:25:20.266795 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7924e0d1-1985-4f6f-8ff6-6b27d641ed54 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:20.362766 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:25:20.365778 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7924e0d1-1985-4f6f-8ff6-6b27d641ed54 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:20.399435 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0849c696-ecd1-4d98-9eb7-eadbbeaaee46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:20.400031 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0849c696-ecd1-4d98-9eb7-eadbbeaaee46 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:20.400306 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0849c696-ecd1-4d98-9eb7-eadbbeaaee46 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:20.400573 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0849c696-ecd1-4d98-9eb7-eadbbeaaee46 None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:20.401217 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0849c696-ecd1-4d98-9eb7-eadbbeaaee46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:25:20.401531 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0849c696-ecd1-4d98-9eb7-eadbbeaaee46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:25:20.402899 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0849c696-ecd1-4d98-9eb7-eadbbeaaee46 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:25:20.413826 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 596/1211] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 24 06:25:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:25:20.419569 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0618d06-de2e-4665-9886-0110a7242a31 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:20.524282 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0618d06-de2e-4665-9886-0110a7242a31 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:20.525794 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:25:20.558639 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8512ae86-4248-498a-9e0c-63ad40e1e5d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:20.559331 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8512ae86-4248-498a-9e0c-63ad40e1e5d1 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:20.559617 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8512ae86-4248-498a-9e0c-63ad40e1e5d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:20.559892 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8512ae86-4248-498a-9e0c-63ad40e1e5d1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:20.567506 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8512ae86-4248-498a-9e0c-63ad40e1e5d1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:25:20.568935 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8512ae86-4248-498a-9e0c-63ad40e1e5d1 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:25:20.580359 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-8512ae86-4248-498a-9e0c-63ad40e1e5d1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 06:25:20.580359 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:25:20.580359 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:25:20.580359 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:25:20.580359 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:25:20.580359 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:25:20.580359 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:25:20.580359 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:25:20.580359 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:25:20.580359 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:25:20.580359 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:25:20.580359 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:25:20.580359 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 06:25:20.580359 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 06:25:20.580359 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 06:25:20.580359 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 06:25:20.581569 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:25:20.581569 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:25:20.581569 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 06:25:20.581569 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:25:20.581569 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 06:25:20.581569 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:25:20.581569 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:25:20.581569 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:25:20.581569 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:25:20.581569 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:25:20.581569 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:25:20.581569 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:25:20.581569 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:25:20.581569 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:25:20.581569 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:25:20.581569 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:25:20.581569 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:25:20.581569 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:25:20.582893 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:25:20.582893 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:25:20.582893 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 06:25:20.582893 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 06:25:20.582893 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 06:25:20.582893 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 06:25:20.582893 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 06:25:20.582893 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 06:25:20.582893 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 06:25:20.582893 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:25:20.582893 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 616/1212] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:25:20 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:25:20.587338 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0849c696-ecd1-4d98-9eb7-eadbbeaaee46 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:20.690338 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0849c696-ecd1-4d98-9eb7-eadbbeaaee46 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:20.724617 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163e0ff6-df2a-4125-b4ea-2d6f45fdd07c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:20.725356 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163e0ff6-df2a-4125-b4ea-2d6f45fdd07c None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:20.725636 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163e0ff6-df2a-4125-b4ea-2d6f45fdd07c None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:20.725905 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163e0ff6-df2a-4125-b4ea-2d6f45fdd07c None admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:20.726674 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163e0ff6-df2a-4125-b4ea-2d6f45fdd07c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:25:20.726975 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163e0ff6-df2a-4125-b4ea-2d6f45fdd07c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:25:20.728379 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163e0ff6-df2a-4125-b4ea-2d6f45fdd07c None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:25:20.746323 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 597/1213] 10.222.0.47 () {56 vars in 1143 bytes} [Wed Aug 24 06:25:20 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:25:20.752386 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8512ae86-4248-498a-9e0c-63ad40e1e5d1 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:20.853613 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8512ae86-4248-498a-9e0c-63ad40e1e5d1 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:20.891884 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72cc8e9-24c7-4122-bd87-ea4b739a0b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:20.892407 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72cc8e9-24c7-4122-bd87-ea4b739a0b48 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:20.892687 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72cc8e9-24c7-4122-bd87-ea4b739a0b48 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:20.892960 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72cc8e9-24c7-4122-bd87-ea4b739a0b48 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:20.923355 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:25:20.962955 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72cc8e9-24c7-4122-bd87-ea4b739a0b48 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:25:20.964421 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72cc8e9-24c7-4122-bd87-ea4b739a0b48 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:25:21.065004 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 617/1214] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 24 06:25:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 06:25:21.071763 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163e0ff6-df2a-4125-b4ea-2d6f45fdd07c None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:21.090903 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:25:21.172578 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163e0ff6-df2a-4125-b4ea-2d6f45fdd07c None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:21.213172 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c16bfd3-4101-4afb-b431-28e6ccb4acca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:21.213907 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c16bfd3-4101-4afb-b431-28e6ccb4acca None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:21.214181 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c16bfd3-4101-4afb-b431-28e6ccb4acca None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:21.214447 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c16bfd3-4101-4afb-b431-28e6ccb4acca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:21.227077 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c16bfd3-4101-4afb-b431-28e6ccb4acca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:25:21.228539 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c16bfd3-4101-4afb-b431-28e6ccb4acca None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:25:21.245898 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-4c16bfd3-4101-4afb-b431-28e6ccb4acca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:25:21.245898 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:25:21.245898 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:25:21.245898 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:25:21.245898 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:25:21.245898 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:25:21.245898 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:25:21.245898 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:25:21.245898 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:25:21.245898 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:25:21.245898 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:25:21.245898 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:25:21.245898 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:25:21.245898 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:25:21.245898 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:25:21.245898 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:25:21.247174 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:25:21.247174 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:25:21.247174 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:25:21.247174 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:25:21.247174 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:25:21.247174 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:25:21.247174 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:25:21.247174 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:25:21.247174 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:25:21.247174 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:25:21.247174 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:25:21.247174 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:25:21.247174 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:25:21.247174 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:25:21.247174 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:25:21.247174 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:25:21.247174 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:25:21.248078 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:25:21.248078 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:25:21.248078 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:25:21.248078 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 598/1215] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:25:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:25:21.253106 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72cc8e9-24c7-4122-bd87-ea4b739a0b48 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:21.318753 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:25:21.358563 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72cc8e9-24c7-4122-bd87-ea4b739a0b48 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:21.392542 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25c516b-86b6-4bcd-afe2-381cc2f587e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:21.393142 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c516b-86b6-4bcd-afe2-381cc2f587e7 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:21.393452 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c516b-86b6-4bcd-afe2-381cc2f587e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:21.393730 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c516b-86b6-4bcd-afe2-381cc2f587e7 None admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:21.395037 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25c516b-86b6-4bcd-afe2-381cc2f587e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:25:21.395514 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25c516b-86b6-4bcd-afe2-381cc2f587e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:25:21.396881 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25c516b-86b6-4bcd-afe2-381cc2f587e7 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:25:21.408787 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 618/1216] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 24 06:25:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:25:21.415451 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c16bfd3-4101-4afb-b431-28e6ccb4acca None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:21.487591 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:25:21.519753 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c16bfd3-4101-4afb-b431-28e6ccb4acca None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:21.555447 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a0f360-2378-4d57-93a9-c02abe71e35c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:21.556069 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a0f360-2378-4d57-93a9-c02abe71e35c None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:21.556352 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a0f360-2378-4d57-93a9-c02abe71e35c None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:21.556626 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a0f360-2378-4d57-93a9-c02abe71e35c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:21.563351 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a0f360-2378-4d57-93a9-c02abe71e35c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:25:21.564796 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a0f360-2378-4d57-93a9-c02abe71e35c None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:25:21.576566 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-02a0f360-2378-4d57-93a9-c02abe71e35c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:25:21.576566 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:25:21.576566 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:25:21.576566 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:25:21.576566 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:25:21.576566 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:25:21.576566 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:25:21.576566 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:25:21.576566 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:25:21.576566 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:25:21.576566 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:25:21.576566 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:25:21.576566 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 06:25:21.576566 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 06:25:21.576566 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 06:25:21.577600 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 06:25:21.577600 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:25:21.577600 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:25:21.577600 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:25:21.577600 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:25:21.577600 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:25:21.577600 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:25:21.577600 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:25:21.577600 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:25:21.577600 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:25:21.577600 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:25:21.577600 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:25:21.577600 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:25:21.577600 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:25:21.577600 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:25:21.577600 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:25:21.577600 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:25:21.578513 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 06:25:21.578513 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 06:25:21.578513 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:25:21.578513 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:25:21.578513 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:25:21.578513 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:25:21.578513 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:25:21.578513 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:25:21.578513 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 599/1217] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 24 06:25:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:25:21.584271 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25c516b-86b6-4bcd-afe2-381cc2f587e7 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:21.686038 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25c516b-86b6-4bcd-afe2-381cc2f587e7 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:21.709076 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:25:21.720281 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4aa1c3-5d15-45b1-ad55-82b83dfa5f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:21.720881 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4aa1c3-5d15-45b1-ad55-82b83dfa5f65 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:21.721243 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4aa1c3-5d15-45b1-ad55-82b83dfa5f65 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:21.721563 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4aa1c3-5d15-45b1-ad55-82b83dfa5f65 None admin] PATH_INFO: `/v3/projects` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:21.722381 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4aa1c3-5d15-45b1-ad55-82b83dfa5f65 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:25:21.722744 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4aa1c3-5d15-45b1-ad55-82b83dfa5f65 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:25:21.724253 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4aa1c3-5d15-45b1-ad55-82b83dfa5f65 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:25:21.735652 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 619/1218] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Aug 24 06:25:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:25:21.741896 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a0f360-2378-4d57-93a9-c02abe71e35c None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:21.843531 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a0f360-2378-4d57-93a9-c02abe71e35c None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:21.871660 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:25:21.877286 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4573a8-205b-476d-88b4-783c9741a0a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:21.877888 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4573a8-205b-476d-88b4-783c9741a0a0 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:21.878167 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4573a8-205b-476d-88b4-783c9741a0a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:21.878968 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4573a8-205b-476d-88b4-783c9741a0a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:21.880009 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4573a8-205b-476d-88b4-783c9741a0a0 None admin] RBAC: Adding query filter params (role.id=1712d85aedaf4189bf4bc89094d888c3, scope.project.id=c02e49a89cdd40d885bfa816f06c58a1, user.id=306188d42dbd4b5b84082ab3842eaf9a) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:25:21.880317 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4573a8-205b-476d-88b4-783c9741a0a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:25:21.881775 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4573a8-205b-476d-88b4-783c9741a0a0 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:25:21.887242 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 600/1219] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 24 06:25:21 2022] GET /identity/v3/role_assignments?user.id=306188d42dbd4b5b84082ab3842eaf9a&scope.project.id=c02e49a89cdd40d885bfa816f06c58a1&role.id=1712d85aedaf4189bf4bc89094d888c3 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:25:23.801069 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483112e1-44ec-46fa-8b85-3a19e56f5e92 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:23.801617 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483112e1-44ec-46fa-8b85-3a19e56f5e92 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:23.801750 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483112e1-44ec-46fa-8b85-3a19e56f5e92 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:23.803033 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 620/1220] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:25:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:25:23.808382 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1851520-a3e5-406d-8913-f86b46302a61 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:23.808951 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1851520-a3e5-406d-8913-f86b46302a61 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:23.808951 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1851520-a3e5-406d-8913-f86b46302a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:23.827596 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-e1851520-a3e5-406d-8913-f86b46302a61 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:25:23.828786 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-e1851520-a3e5-406d-8913-f86b46302a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:25:23.898454 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 601/1221] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:25:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:25:24.173459 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483112e1-44ec-46fa-8b85-3a19e56f5e92 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:24.275315 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483112e1-44ec-46fa-8b85-3a19e56f5e92 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:24.309329 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92a0ef1-c4c7-471a-bbba-1ef8b0482632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:24.309948 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92a0ef1-c4c7-471a-bbba-1ef8b0482632 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:24.310223 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92a0ef1-c4c7-471a-bbba-1ef8b0482632 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:24.310488 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92a0ef1-c4c7-471a-bbba-1ef8b0482632 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:24.320889 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92a0ef1-c4c7-471a-bbba-1ef8b0482632 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:25:24.325571 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-a92a0ef1-c4c7-471a-bbba-1ef8b0482632 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:25:24.325571 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:25:24.325571 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:25:24.325571 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:25:24.325571 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:25:24.325571 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:25:24.325571 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:25:24.325571 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:25:24.325571 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:25:24.325571 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:25:24.325571 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:25:24.325571 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:25:24.325571 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:25:24.325571 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:25:24.325571 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:25:24.325571 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:25:24.326666 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:25:24.326666 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:25:24.326666 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:25:24.326666 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:25:24.326666 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:25:24.326666 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:25:24.326666 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:25:24.326666 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:25:24.326666 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:25:24.326666 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:25:24.326666 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:25:24.326666 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:25:24.326666 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:25:24.326666 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:25:24.326666 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:25:24.326666 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:25:24.327704 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:25:24.327704 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:25:24.327704 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:25:24.327704 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:25:24.327704 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:25:24.327704 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:25:24.327704 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:25:24.327704 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:25:24.327704 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:25:24.327704 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:25:24.327704 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:25:24.327704 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:25:24.327704 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 621/1222] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Aug 24 06:25:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:25:24.332884 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1851520-a3e5-406d-8913-f86b46302a61 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:24.432097 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1851520-a3e5-406d-8913-f86b46302a61 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:24.454743 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:25:24.465417 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3718c6e8-e428-470b-b350-5f390a67fc5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:24.466058 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3718c6e8-e428-470b-b350-5f390a67fc5d None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:24.466340 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3718c6e8-e428-470b-b350-5f390a67fc5d None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:24.466611 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3718c6e8-e428-470b-b350-5f390a67fc5d None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:24.467275 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3718c6e8-e428-470b-b350-5f390a67fc5d None admin] RBAC: Adding query filter params (name=service) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:25:24.467577 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3718c6e8-e428-470b-b350-5f390a67fc5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:25:24.468927 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3718c6e8-e428-470b-b350-5f390a67fc5d None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:25:24.476740 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 602/1223] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Aug 24 06:25:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:25:24.482755 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92a0ef1-c4c7-471a-bbba-1ef8b0482632 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:24.527601 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:25:24.584512 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92a0ef1-c4c7-471a-bbba-1ef8b0482632 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:24.618583 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231b4249-1265-49ba-80ec-28c7001c8a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:24.619219 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231b4249-1265-49ba-80ec-28c7001c8a52 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:24.619498 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231b4249-1265-49ba-80ec-28c7001c8a52 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:24.619772 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231b4249-1265-49ba-80ec-28c7001c8a52 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:24.631990 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231b4249-1265-49ba-80ec-28c7001c8a52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:25:24.633461 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231b4249-1265-49ba-80ec-28c7001c8a52 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:25:24.648112 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-231b4249-1265-49ba-80ec-28c7001c8a52 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:25:24.648112 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:25:24.648112 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:25:24.648112 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:25:24.648112 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:25:24.648112 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:25:24.648112 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:25:24.648112 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:25:24.648112 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:25:24.648112 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:25:24.648112 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:25:24.648112 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:25:24.648112 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:25:24.648112 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:25:24.648112 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:25:24.648112 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:25:24.654182 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:25:24.654182 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:25:24.654182 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:25:24.654182 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:25:24.654182 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:25:24.654182 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:25:24.654182 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:25:24.654182 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:25:24.654182 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:25:24.654182 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:25:24.654182 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:25:24.654182 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:25:24.654182 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:25:24.654182 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:25:24.654182 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:25:24.654182 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:25:24.654182 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:25:24.655325 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:25:24.655325 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:25:24.655325 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:25:24.655325 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 622/1224] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:25:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:25:24.655579 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3718c6e8-e428-470b-b350-5f390a67fc5d None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:24.756396 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3718c6e8-e428-470b-b350-5f390a67fc5d None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:24.789912 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dd54c1-4805-4287-bdfa-76e28fe63e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:24.790516 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dd54c1-4805-4287-bdfa-76e28fe63e00 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:24.790799 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dd54c1-4805-4287-bdfa-76e28fe63e00 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:24.791080 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dd54c1-4805-4287-bdfa-76e28fe63e00 None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:24.791722 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dd54c1-4805-4287-bdfa-76e28fe63e00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:25:24.792025 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dd54c1-4805-4287-bdfa-76e28fe63e00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:25:24.793409 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dd54c1-4805-4287-bdfa-76e28fe63e00 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:25:24.805665 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 603/1225] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 24 06:25:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:25:24.811427 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231b4249-1265-49ba-80ec-28c7001c8a52 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:24.904534 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:25:24.911548 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231b4249-1265-49ba-80ec-28c7001c8a52 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:24.945325 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0111b246-6fc8-4a3f-9304-f8c8038bb33d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:24.945950 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0111b246-6fc8-4a3f-9304-f8c8038bb33d None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:24.946241 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0111b246-6fc8-4a3f-9304-f8c8038bb33d None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:24.946514 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0111b246-6fc8-4a3f-9304-f8c8038bb33d None admin] PATH_INFO: `/v3/users/placement` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:24.953264 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0111b246-6fc8-4a3f-9304-f8c8038bb33d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:25:24.954705 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0111b246-6fc8-4a3f-9304-f8c8038bb33d None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:25:24.965908 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-0111b246-6fc8-4a3f-9304-f8c8038bb33d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 06:25:24.965908 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:25:24.965908 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:25:24.965908 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:25:24.965908 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:25:24.965908 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:25:24.965908 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:25:24.965908 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:25:24.965908 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:25:24.965908 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:25:24.965908 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:25:24.965908 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:25:24.965908 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 06:25:24.965908 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 06:25:24.965908 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 06:25:24.965908 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 06:25:24.967084 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:25:24.967084 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:25:24.967084 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 06:25:24.967084 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:25:24.967084 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 06:25:24.967084 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:25:24.967084 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:25:24.967084 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:25:24.967084 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:25:24.967084 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:25:24.967084 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:25:24.967084 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:25:24.967084 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:25:24.967084 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:25:24.967084 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:25:24.967084 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:25:24.967084 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:25:24.967084 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:25:24.968077 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:25:24.968077 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:25:24.968077 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 06:25:24.968077 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 06:25:24.968077 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 06:25:24.968077 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 06:25:24.968077 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 06:25:24.968077 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 06:25:24.968077 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 06:25:24.968077 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:25:24.968077 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 623/1226] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:25:24 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:25:24.973904 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dd54c1-4805-4287-bdfa-76e28fe63e00 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:25.021237 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:25:25.075825 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dd54c1-4805-4287-bdfa-76e28fe63e00 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:25.113628 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff7fd74-9436-43f5-a756-9be50b88ac69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:25.114226 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff7fd74-9436-43f5-a756-9be50b88ac69 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:25.114500 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff7fd74-9436-43f5-a756-9be50b88ac69 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:25.114770 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff7fd74-9436-43f5-a756-9be50b88ac69 None admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:25.115489 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff7fd74-9436-43f5-a756-9be50b88ac69 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:25:25.115788 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff7fd74-9436-43f5-a756-9be50b88ac69 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:25:25.117219 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff7fd74-9436-43f5-a756-9be50b88ac69 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:25:25.134440 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 604/1227] 10.222.0.47 () {56 vars in 1143 bytes} [Wed Aug 24 06:25:24 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:25:25.140165 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0111b246-6fc8-4a3f-9304-f8c8038bb33d None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:25.241010 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0111b246-6fc8-4a3f-9304-f8c8038bb33d None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:25.275743 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daa3610-beca-4c31-9e8d-27f5ecec95d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:25.276358 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daa3610-beca-4c31-9e8d-27f5ecec95d8 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:25.276576 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daa3610-beca-4c31-9e8d-27f5ecec95d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:25.276853 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daa3610-beca-4c31-9e8d-27f5ecec95d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:25.287877 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daa3610-beca-4c31-9e8d-27f5ecec95d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:25:25.289425 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4daa3610-beca-4c31-9e8d-27f5ecec95d8 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:25:25.303028 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-4daa3610-beca-4c31-9e8d-27f5ecec95d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:25:25.303028 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:25:25.303028 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:25:25.303028 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:25:25.303028 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:25:25.303028 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:25:25.303028 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:25:25.303028 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:25:25.303028 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:25:25.303028 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:25:25.303028 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:25:25.303028 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:25:25.303028 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:25:25.303028 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:25:25.303028 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:25:25.303028 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:25:25.309091 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:25:25.309091 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:25:25.309091 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:25:25.309091 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:25:25.309091 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:25:25.309091 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:25:25.309091 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:25:25.309091 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:25:25.309091 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:25:25.309091 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:25:25.309091 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:25:25.309091 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:25:25.309091 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:25:25.309091 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:25:25.309091 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:25:25.309091 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:25:25.309091 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:25:25.310326 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:25:25.310326 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:25:25.310326 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:25:25.310326 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 624/1228] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:25:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:25:25.310640 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff7fd74-9436-43f5-a756-9be50b88ac69 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:25.407251 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff7fd74-9436-43f5-a756-9be50b88ac69 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:25.410427 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:25:25.441075 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca44be2-0e43-480f-9d1c-1ca915e13114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:25.441796 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca44be2-0e43-480f-9d1c-1ca915e13114 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:25.442075 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca44be2-0e43-480f-9d1c-1ca915e13114 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:25.442351 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca44be2-0e43-480f-9d1c-1ca915e13114 None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:25.442983 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca44be2-0e43-480f-9d1c-1ca915e13114 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:25:25.443284 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca44be2-0e43-480f-9d1c-1ca915e13114 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:25:25.444684 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca44be2-0e43-480f-9d1c-1ca915e13114 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:25:25.456035 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 605/1229] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 24 06:25:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:25:25.462342 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daa3610-beca-4c31-9e8d-27f5ecec95d8 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:25.471834 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:25:25.560899 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daa3610-beca-4c31-9e8d-27f5ecec95d8 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:25.594280 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745ea893-85c0-46cf-973f-1971344e230f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:25.594895 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745ea893-85c0-46cf-973f-1971344e230f None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:25.595171 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745ea893-85c0-46cf-973f-1971344e230f None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:25.595441 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745ea893-85c0-46cf-973f-1971344e230f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:25.602026 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745ea893-85c0-46cf-973f-1971344e230f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:25:25.603435 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745ea893-85c0-46cf-973f-1971344e230f None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:25:25.615460 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-745ea893-85c0-46cf-973f-1971344e230f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:25:25.615460 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:25:25.615460 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:25:25.615460 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:25:25.615460 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:25:25.615460 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:25:25.615460 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:25:25.615460 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:25:25.615460 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:25:25.615460 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:25:25.615460 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:25:25.615460 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:25:25.615460 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 06:25:25.615460 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 06:25:25.615460 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 06:25:25.617359 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 06:25:25.617359 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:25:25.617359 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:25:25.617359 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:25:25.617359 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:25:25.617359 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:25:25.617359 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:25:25.617359 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:25:25.617359 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:25:25.617359 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:25:25.617359 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:25:25.617359 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:25:25.617359 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:25:25.617359 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:25:25.617359 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:25:25.617359 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:25:25.617359 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:25:25.618642 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 06:25:25.618642 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 06:25:25.618642 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:25:25.618642 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:25:25.618642 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:25:25.618642 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:25:25.618642 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:25:25.618642 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:25:25.618642 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 625/1230] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 24 06:25:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:25:25.622629 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca44be2-0e43-480f-9d1c-1ca915e13114 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:25.725788 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca44be2-0e43-480f-9d1c-1ca915e13114 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:25.759732 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcfb2f1-561a-4801-b81b-89eefe8e743d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:25.760339 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcfb2f1-561a-4801-b81b-89eefe8e743d None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:25.760617 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcfb2f1-561a-4801-b81b-89eefe8e743d None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:25.760905 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcfb2f1-561a-4801-b81b-89eefe8e743d None admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:25.761628 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcfb2f1-561a-4801-b81b-89eefe8e743d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:25:25.761933 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcfb2f1-561a-4801-b81b-89eefe8e743d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:25:25.763379 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcfb2f1-561a-4801-b81b-89eefe8e743d None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:25:25.776624 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 606/1231] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Aug 24 06:25:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:25:25.782819 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745ea893-85c0-46cf-973f-1971344e230f None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:25.848252 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:25:25.883337 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745ea893-85c0-46cf-973f-1971344e230f None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:25.915448 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939017db-c34c-4989-a45e-e253a9c17f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:25.916088 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939017db-c34c-4989-a45e-e253a9c17f6a None admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:25.916363 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939017db-c34c-4989-a45e-e253a9c17f6a None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:25.916631 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939017db-c34c-4989-a45e-e253a9c17f6a None admin] PATH_INFO: `/v3/projects/c02e49a89cdd40d885bfa816f06c58a1/users/306188d42dbd4b5b84082ab3842eaf9a/roles/1712d85aedaf4189bf4bc89094d888c3` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:25.931484 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939017db-c34c-4989-a45e-e253a9c17f6a None admin] RBAC: Authorizing `identity:create_grant(project_id=c02e49a89cdd40d885bfa816f06c58a1, user_id=306188d42dbd4b5b84082ab3842eaf9a, role_id=1712d85aedaf4189bf4bc89094d888c3)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:25:25.932915 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939017db-c34c-4989-a45e-e253a9c17f6a None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:25:26.019727 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 626/1232] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Aug 24 06:25:25 2022] PUT /identity/v3/projects/c02e49a89cdd40d885bfa816f06c58a1/users/306188d42dbd4b5b84082ab3842eaf9a/roles/1712d85aedaf4189bf4bc89094d888c3 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:25:27.963385 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e69e6d-944f-42c9-a12a-8afd9e53c66e None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:27.963910 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e69e6d-944f-42c9-a12a-8afd9e53c66e None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:27.963910 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e69e6d-944f-42c9-a12a-8afd9e53c66e None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:27.965160 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 607/1233] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:25:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:25:27.970405 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1ddfc7-a1ce-4392-b9a4-ab6c8e580ecb None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:27.970787 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1ddfc7-a1ce-4392-b9a4-ab6c8e580ecb None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:27.970953 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1ddfc7-a1ce-4392-b9a4-ab6c8e580ecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:27.989012 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-fe1ddfc7-a1ce-4392-b9a4-ab6c8e580ecb None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:25:27.990259 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-fe1ddfc7-a1ce-4392-b9a4-ab6c8e580ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:25:28.057736 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 627/1234] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:25:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:25:28.326595 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e69e6d-944f-42c9-a12a-8afd9e53c66e None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:28.350797 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:25:28.423228 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e69e6d-944f-42c9-a12a-8afd9e53c66e None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:28.456050 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a08dd36-8d19-4e9b-bc3e-4173df8b0163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:28.456666 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a08dd36-8d19-4e9b-bc3e-4173df8b0163 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:28.456940 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a08dd36-8d19-4e9b-bc3e-4173df8b0163 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:28.457231 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a08dd36-8d19-4e9b-bc3e-4173df8b0163 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:28.467925 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a08dd36-8d19-4e9b-bc3e-4173df8b0163 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:25:28.473824 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-5a08dd36-8d19-4e9b-bc3e-4173df8b0163 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:25:28.473824 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:25:28.473824 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:25:28.473824 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:25:28.473824 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:25:28.473824 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:25:28.473824 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:25:28.473824 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:25:28.473824 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:25:28.473824 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:25:28.473824 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:25:28.473824 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:25:28.473824 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:25:28.473824 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:25:28.473824 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:25:28.473824 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:25:28.474926 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:25:28.474926 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:25:28.474926 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:25:28.474926 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:25:28.474926 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:25:28.474926 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:25:28.474926 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:25:28.474926 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:25:28.474926 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:25:28.474926 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:25:28.474926 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:25:28.474926 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:25:28.474926 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:25:28.474926 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:25:28.474926 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:25:28.474926 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:25:28.476057 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:25:28.476057 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:25:28.476057 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:25:28.476057 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:25:28.476057 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:25:28.476057 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:25:28.476057 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:25:28.476057 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:25:28.476057 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:25:28.476057 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:25:28.476057 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 06:25:28.476057 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:25:28.476057 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 608/1235] 10.222.0.47 () {56 vars in 1102 bytes} [Wed Aug 24 06:25:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:25:28.480949 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1ddfc7-a1ce-4392-b9a4-ab6c8e580ecb None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:28.527482 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:25:28.581455 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1ddfc7-a1ce-4392-b9a4-ab6c8e580ecb None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:28.616180 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cef47a-5c9b-4325-a1a6-838d240d87ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:28.616787 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cef47a-5c9b-4325-a1a6-838d240d87ce None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:28.617068 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cef47a-5c9b-4325-a1a6-838d240d87ce None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:28.617394 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cef47a-5c9b-4325-a1a6-838d240d87ce None admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:28.618065 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cef47a-5c9b-4325-a1a6-838d240d87ce None admin] RBAC: Adding query filter params (name=service) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:25:28.618369 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cef47a-5c9b-4325-a1a6-838d240d87ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:25:28.619749 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cef47a-5c9b-4325-a1a6-838d240d87ce None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:25:28.627959 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 628/1236] 10.222.0.47 () {56 vars in 1103 bytes} [Wed Aug 24 06:25:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:25:28.634387 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a08dd36-8d19-4e9b-bc3e-4173df8b0163 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:28.723898 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:25:28.739653 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a08dd36-8d19-4e9b-bc3e-4173df8b0163 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:28.773800 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f368c1-a759-4a51-bf8e-aa81f47e0339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:28.774429 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f368c1-a759-4a51-bf8e-aa81f47e0339 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:28.774721 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f368c1-a759-4a51-bf8e-aa81f47e0339 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:28.775004 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f368c1-a759-4a51-bf8e-aa81f47e0339 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:28.785275 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f368c1-a759-4a51-bf8e-aa81f47e0339 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:25:28.786735 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f368c1-a759-4a51-bf8e-aa81f47e0339 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:25:28.802279 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-25f368c1-a759-4a51-bf8e-aa81f47e0339 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:25:28.802279 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:25:28.802279 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:25:28.802279 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:25:28.802279 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:25:28.802279 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:25:28.802279 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:25:28.802279 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:25:28.802279 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:25:28.802279 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:25:28.802279 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:25:28.802279 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:25:28.802279 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:25:28.802279 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:25:28.802279 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:25:28.802279 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:25:28.803456 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:25:28.803456 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:25:28.803456 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:25:28.803456 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:25:28.803456 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:25:28.803456 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:25:28.803456 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:25:28.803456 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:25:28.803456 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:25:28.803456 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:25:28.803456 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:25:28.803456 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:25:28.803456 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:25:28.803456 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:25:28.803456 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:25:28.803456 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:25:28.803456 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:25:28.804868 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:25:28.804868 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:25:28.804868 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:25:28.804868 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 609/1237] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:25:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:25:28.809768 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cef47a-5c9b-4325-a1a6-838d240d87ce None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:28.914935 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cef47a-5c9b-4325-a1a6-838d240d87ce None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:28.924230 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:25:28.949544 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfce8dc8-8bfd-4cf3-a93d-a7662574266c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:28.950157 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfce8dc8-8bfd-4cf3-a93d-a7662574266c None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:28.950441 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfce8dc8-8bfd-4cf3-a93d-a7662574266c None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:28.950754 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfce8dc8-8bfd-4cf3-a93d-a7662574266c None admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:28.951388 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfce8dc8-8bfd-4cf3-a93d-a7662574266c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:25:28.951716 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfce8dc8-8bfd-4cf3-a93d-a7662574266c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:25:28.953086 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfce8dc8-8bfd-4cf3-a93d-a7662574266c None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:25:28.965744 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 629/1238] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 24 06:25:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:25:28.972447 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f368c1-a759-4a51-bf8e-aa81f47e0339 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:29.091693 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f368c1-a759-4a51-bf8e-aa81f47e0339 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:29.132385 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deef3349-ce31-4912-87b0-46b227ce2531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:29.133017 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deef3349-ce31-4912-87b0-46b227ce2531 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:29.133442 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deef3349-ce31-4912-87b0-46b227ce2531 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:29.133718 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deef3349-ce31-4912-87b0-46b227ce2531 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:29.136616 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:25:29.141881 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deef3349-ce31-4912-87b0-46b227ce2531 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:25:29.144541 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deef3349-ce31-4912-87b0-46b227ce2531 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:25:29.157449 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-deef3349-ce31-4912-87b0-46b227ce2531 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 06:25:29.157449 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:25:29.157449 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:25:29.157449 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:25:29.157449 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:25:29.157449 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:25:29.157449 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:25:29.157449 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:25:29.157449 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:25:29.157449 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:25:29.157449 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:25:29.157449 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:25:29.157449 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 06:25:29.157449 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 06:25:29.157449 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 06:25:29.157449 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 06:25:29.161098 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:25:29.161098 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:25:29.161098 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 06:25:29.161098 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:25:29.161098 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 06:25:29.161098 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:25:29.161098 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:25:29.161098 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:25:29.161098 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:25:29.161098 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:25:29.161098 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:25:29.161098 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:25:29.161098 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:25:29.161098 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:25:29.161098 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:25:29.161098 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:25:29.161098 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:25:29.161098 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:25:29.162101 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:25:29.162101 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:25:29.162101 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 06:25:29.162101 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 06:25:29.162101 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 06:25:29.162101 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 06:25:29.162101 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 06:25:29.162101 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 06:25:29.162101 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 06:25:29.162101 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:25:29.162101 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 610/1239] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:25:28 2022] GET /identity/v3/users/placement => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:25:29.166203 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfce8dc8-8bfd-4cf3-a93d-a7662574266c None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:29.272171 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfce8dc8-8bfd-4cf3-a93d-a7662574266c None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:29.307613 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0777077-6c63-4a42-a892-be4185eea323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:29.308225 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0777077-6c63-4a42-a892-be4185eea323 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:29.308510 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0777077-6c63-4a42-a892-be4185eea323 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:29.308787 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0777077-6c63-4a42-a892-be4185eea323 None admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:29.309555 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0777077-6c63-4a42-a892-be4185eea323 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:25:29.309871 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0777077-6c63-4a42-a892-be4185eea323 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:25:29.311316 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0777077-6c63-4a42-a892-be4185eea323 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:25:29.329536 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 630/1240] 10.222.0.47 () {56 vars in 1143 bytes} [Wed Aug 24 06:25:29 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:25:29.337022 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deef3349-ce31-4912-87b0-46b227ce2531 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:29.445124 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deef3349-ce31-4912-87b0-46b227ce2531 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:29.485726 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49efc1c4-c6c4-49b1-888a-204928de2d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:29.486337 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49efc1c4-c6c4-49b1-888a-204928de2d38 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:29.486618 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49efc1c4-c6c4-49b1-888a-204928de2d38 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:29.486894 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49efc1c4-c6c4-49b1-888a-204928de2d38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:29.554437 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:25:29.560841 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49efc1c4-c6c4-49b1-888a-204928de2d38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:25:29.562372 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49efc1c4-c6c4-49b1-888a-204928de2d38 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:25:29.661414 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 611/1241] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 24 06:25:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 06:25:29.667208 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0777077-6c63-4a42-a892-be4185eea323 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:29.694184 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:25:29.767944 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0777077-6c63-4a42-a892-be4185eea323 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:29.808400 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a7a670-e0eb-47e2-b99f-4537e0735b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:29.809042 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a7a670-e0eb-47e2-b99f-4537e0735b0b None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:29.809354 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a7a670-e0eb-47e2-b99f-4537e0735b0b None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:29.809627 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a7a670-e0eb-47e2-b99f-4537e0735b0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:29.822480 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a7a670-e0eb-47e2-b99f-4537e0735b0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:25:29.823971 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a7a670-e0eb-47e2-b99f-4537e0735b0b None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:25:29.839651 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-27a7a670-e0eb-47e2-b99f-4537e0735b0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:25:29.839651 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:25:29.839651 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:25:29.839651 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:25:29.839651 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:25:29.839651 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:25:29.839651 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:25:29.839651 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:25:29.839651 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:25:29.839651 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:25:29.839651 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:25:29.839651 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:25:29.839651 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:25:29.839651 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:25:29.839651 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:25:29.839651 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:25:29.840853 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:25:29.840853 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:25:29.840853 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:25:29.840853 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:25:29.840853 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:25:29.840853 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:25:29.840853 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:25:29.840853 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:25:29.840853 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:25:29.840853 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:25:29.840853 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:25:29.840853 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:25:29.840853 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:25:29.840853 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:25:29.840853 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:25:29.840853 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:25:29.840853 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:25:29.842004 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:25:29.842004 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:25:29.842004 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:25:29.842004 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 631/1242] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:25:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:25:29.851434 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49efc1c4-c6c4-49b1-888a-204928de2d38 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:29.947537 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:25:29.953154 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49efc1c4-c6c4-49b1-888a-204928de2d38 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:29.986088 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bae6f5d-3012-4b1a-a918-b03ed5a0bd28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:29.986738 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bae6f5d-3012-4b1a-a918-b03ed5a0bd28 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:29.986984 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bae6f5d-3012-4b1a-a918-b03ed5a0bd28 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:29.987283 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bae6f5d-3012-4b1a-a918-b03ed5a0bd28 None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:29.987962 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bae6f5d-3012-4b1a-a918-b03ed5a0bd28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:25:29.988295 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bae6f5d-3012-4b1a-a918-b03ed5a0bd28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:25:29.989674 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bae6f5d-3012-4b1a-a918-b03ed5a0bd28 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:25:30.001489 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 612/1243] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 24 06:25:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:25:30.008050 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a7a670-e0eb-47e2-b99f-4537e0735b0b None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:30.088912 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:25:30.110346 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a7a670-e0eb-47e2-b99f-4537e0735b0b None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:30.144951 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e79ab8-6571-4521-b0ef-09b5b6769b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:30.145603 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e79ab8-6571-4521-b0ef-09b5b6769b8c None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:30.145883 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e79ab8-6571-4521-b0ef-09b5b6769b8c None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:30.146151 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e79ab8-6571-4521-b0ef-09b5b6769b8c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:30.152669 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e79ab8-6571-4521-b0ef-09b5b6769b8c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:25:30.154162 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e79ab8-6571-4521-b0ef-09b5b6769b8c None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:25:30.165051 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-72e79ab8-6571-4521-b0ef-09b5b6769b8c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:25:30.165051 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:25:30.165051 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:25:30.165051 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:25:30.165051 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:25:30.165051 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:25:30.165051 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:25:30.165051 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:25:30.165051 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:25:30.165051 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:25:30.165051 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:25:30.165051 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:25:30.165051 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 06:25:30.165051 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 06:25:30.165051 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 06:25:30.166150 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 06:25:30.166150 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:25:30.166150 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:25:30.166150 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:25:30.166150 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:25:30.166150 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:25:30.166150 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:25:30.166150 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:25:30.166150 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:25:30.166150 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:25:30.166150 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:25:30.166150 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:25:30.166150 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:25:30.166150 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:25:30.166150 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:25:30.166150 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:25:30.166150 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:25:30.167645 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 06:25:30.167645 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 06:25:30.167645 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:25:30.167645 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:25:30.167645 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:25:30.167645 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:25:30.167645 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:25:30.167645 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:25:30.167645 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 632/1244] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 24 06:25:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:25:30.172277 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bae6f5d-3012-4b1a-a918-b03ed5a0bd28 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:30.277167 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bae6f5d-3012-4b1a-a918-b03ed5a0bd28 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:30.310465 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e2be85-ad84-4148-85c8-7e4fb401e752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:30.311069 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e2be85-ad84-4148-85c8-7e4fb401e752 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:30.311352 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e2be85-ad84-4148-85c8-7e4fb401e752 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:30.311628 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e2be85-ad84-4148-85c8-7e4fb401e752 None admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:30.312337 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e2be85-ad84-4148-85c8-7e4fb401e752 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:25:30.312646 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e2be85-ad84-4148-85c8-7e4fb401e752 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:25:30.314096 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e2be85-ad84-4148-85c8-7e4fb401e752 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:25:30.326629 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 613/1245] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Aug 24 06:25:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:25:30.332936 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e79ab8-6571-4521-b0ef-09b5b6769b8c None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:30.431916 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e79ab8-6571-4521-b0ef-09b5b6769b8c None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:30.464809 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9878aa73-698c-4c34-9d7b-84a6029e1e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:30.465434 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9878aa73-698c-4c34-9d7b-84a6029e1e03 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:30.465724 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9878aa73-698c-4c34-9d7b-84a6029e1e03 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:30.465997 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9878aa73-698c-4c34-9d7b-84a6029e1e03 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:30.466719 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9878aa73-698c-4c34-9d7b-84a6029e1e03 None admin] RBAC: Adding query filter params (role.id=1712d85aedaf4189bf4bc89094d888c3, scope.project.id=c02e49a89cdd40d885bfa816f06c58a1, user.id=306188d42dbd4b5b84082ab3842eaf9a) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:25:30.467025 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9878aa73-698c-4c34-9d7b-84a6029e1e03 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:25:30.468456 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9878aa73-698c-4c34-9d7b-84a6029e1e03 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:25:30.474251 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 633/1246] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 24 06:25:30 2022] GET /identity/v3/role_assignments?user.id=306188d42dbd4b5b84082ab3842eaf9a&scope.project.id=c02e49a89cdd40d885bfa816f06c58a1&role.id=1712d85aedaf4189bf4bc89094d888c3 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:25:32.458710 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34274049-b861-4dcc-9b32-ad51655dc709 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:32.459284 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34274049-b861-4dcc-9b32-ad51655dc709 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:32.459284 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34274049-b861-4dcc-9b32-ad51655dc709 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:32.460601 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 614/1247] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:25:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:25:32.465935 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74c05fc-5f08-4a6b-af71-af548a715cbe None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:32.466349 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74c05fc-5f08-4a6b-af71-af548a715cbe None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:32.466508 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74c05fc-5f08-4a6b-af71-af548a715cbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:32.473389 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:25:32.485226 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-e74c05fc-5f08-4a6b-af71-af548a715cbe None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:25:32.486392 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-e74c05fc-5f08-4a6b-af71-af548a715cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:25:32.552064 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 634/1248] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:25:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:25:32.822049 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34274049-b861-4dcc-9b32-ad51655dc709 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:32.840715 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:25:32.924176 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34274049-b861-4dcc-9b32-ad51655dc709 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:32.958383 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0699e52e-37c0-4119-a721-96386d0bb2dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:32.959006 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0699e52e-37c0-4119-a721-96386d0bb2dc None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:32.959286 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0699e52e-37c0-4119-a721-96386d0bb2dc None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:32.959560 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0699e52e-37c0-4119-a721-96386d0bb2dc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:32.969542 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0699e52e-37c0-4119-a721-96386d0bb2dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:25:32.974891 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-0699e52e-37c0-4119-a721-96386d0bb2dc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:25:32.974891 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:25:32.974891 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:25:32.974891 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:25:32.974891 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:25:32.974891 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:25:32.974891 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:25:32.974891 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:25:32.974891 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:25:32.974891 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:25:32.974891 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:25:32.974891 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:25:32.974891 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:25:32.974891 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:25:32.974891 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:25:32.974891 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:25:32.976159 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:25:32.976159 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:25:32.976159 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:25:32.976159 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:25:32.976159 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:25:32.976159 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:25:32.976159 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:25:32.976159 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:25:32.976159 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:25:32.976159 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:25:32.976159 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:25:32.976159 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:25:32.976159 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:25:32.976159 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:25:32.976159 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:25:32.976159 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:25:32.977085 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:25:32.977085 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:25:32.977085 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:25:32.977085 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:25:32.977085 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:25:32.977085 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:25:32.977085 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:25:32.977085 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:25:32.977085 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:25:32.977085 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:25:32.977085 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:25:32.977085 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:25:32.977085 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 615/1249] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Aug 24 06:25:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:25:32.982329 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74c05fc-5f08-4a6b-af71-af548a715cbe None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:33.082895 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74c05fc-5f08-4a6b-af71-af548a715cbe None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:33.111369 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:25:33.116844 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4794030e-1218-4750-93a9-350c46e1927c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:33.117483 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4794030e-1218-4750-93a9-350c46e1927c None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:33.117770 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4794030e-1218-4750-93a9-350c46e1927c None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:33.118042 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4794030e-1218-4750-93a9-350c46e1927c None admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:33.118701 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4794030e-1218-4750-93a9-350c46e1927c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:25:33.119002 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4794030e-1218-4750-93a9-350c46e1927c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:25:33.120361 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4794030e-1218-4750-93a9-350c46e1927c None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:25:33.129144 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 635/1250] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 24 06:25:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:25:33.135290 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0699e52e-37c0-4119-a721-96386d0bb2dc None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:33.214016 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:25:33.235678 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0699e52e-37c0-4119-a721-96386d0bb2dc None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:33.270332 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aa7b16-b7ad-49e1-8ed9-60573310eaa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:33.270913 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aa7b16-b7ad-49e1-8ed9-60573310eaa2 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:33.271192 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aa7b16-b7ad-49e1-8ed9-60573310eaa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:33.271463 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aa7b16-b7ad-49e1-8ed9-60573310eaa2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:33.281383 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32aa7b16-b7ad-49e1-8ed9-60573310eaa2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:25:33.282811 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32aa7b16-b7ad-49e1-8ed9-60573310eaa2 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:25:33.297604 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-32aa7b16-b7ad-49e1-8ed9-60573310eaa2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:25:33.297604 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:25:33.297604 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:25:33.297604 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:25:33.297604 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:25:33.297604 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:25:33.297604 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:25:33.297604 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:25:33.297604 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:25:33.297604 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:25:33.297604 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:25:33.297604 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:25:33.297604 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:25:33.297604 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:25:33.297604 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:25:33.297604 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:25:33.298785 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:25:33.298785 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:25:33.298785 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:25:33.298785 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:25:33.298785 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:25:33.298785 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:25:33.298785 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:25:33.298785 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:25:33.298785 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:25:33.298785 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:25:33.298785 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:25:33.298785 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:25:33.298785 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:25:33.298785 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:25:33.298785 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:25:33.298785 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:25:33.298785 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:25:33.300732 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:25:33.300732 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:25:33.300732 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:25:33.300732 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 616/1251] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:25:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:25:33.305390 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4794030e-1218-4750-93a9-350c46e1927c None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:33.406479 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4794030e-1218-4750-93a9-350c46e1927c None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:33.442160 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8501b9-83f6-4b7f-844e-8e0d154149e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:33.442759 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8501b9-83f6-4b7f-844e-8e0d154149e4 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:33.443042 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8501b9-83f6-4b7f-844e-8e0d154149e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:33.443333 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8501b9-83f6-4b7f-844e-8e0d154149e4 None admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:33.443958 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8501b9-83f6-4b7f-844e-8e0d154149e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:25:33.444259 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8501b9-83f6-4b7f-844e-8e0d154149e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:25:33.445654 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8501b9-83f6-4b7f-844e-8e0d154149e4 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:25:33.457442 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 636/1252] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 24 06:25:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:25:33.463174 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aa7b16-b7ad-49e1-8ed9-60573310eaa2 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:33.561875 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aa7b16-b7ad-49e1-8ed9-60573310eaa2 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:33.589148 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:25:33.597399 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f982ed-e769-4982-b1c3-02c3a8d68337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:33.598018 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f982ed-e769-4982-b1c3-02c3a8d68337 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:33.598294 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f982ed-e769-4982-b1c3-02c3a8d68337 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:33.598564 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f982ed-e769-4982-b1c3-02c3a8d68337 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:33.605834 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f982ed-e769-4982-b1c3-02c3a8d68337 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:25:33.607245 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f982ed-e769-4982-b1c3-02c3a8d68337 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:25:33.618600 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-12f982ed-e769-4982-b1c3-02c3a8d68337 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 06:25:33.618600 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:25:33.618600 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:25:33.618600 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:25:33.618600 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:25:33.618600 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:25:33.618600 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:25:33.618600 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:25:33.618600 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:25:33.618600 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:25:33.618600 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:25:33.618600 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:25:33.618600 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 06:25:33.618600 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 06:25:33.618600 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 06:25:33.618600 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 06:25:33.619754 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:25:33.619754 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:25:33.619754 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 06:25:33.619754 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:25:33.619754 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 06:25:33.619754 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:25:33.619754 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:25:33.619754 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:25:33.619754 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:25:33.619754 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:25:33.619754 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:25:33.619754 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:25:33.619754 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:25:33.619754 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:25:33.619754 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:25:33.619754 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:25:33.619754 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:25:33.619754 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:25:33.620821 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:25:33.620821 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:25:33.620821 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 06:25:33.620821 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 06:25:33.620821 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 06:25:33.620821 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 06:25:33.620821 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 06:25:33.620821 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 06:25:33.620821 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 06:25:33.620821 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:25:33.620821 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 617/1253] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:25:33 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:25:33.625681 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8501b9-83f6-4b7f-844e-8e0d154149e4 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:33.684178 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:25:33.727132 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8501b9-83f6-4b7f-844e-8e0d154149e4 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:33.760687 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab41e73-6b5c-4a19-928b-1b9023991538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:33.761311 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab41e73-6b5c-4a19-928b-1b9023991538 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:33.761590 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab41e73-6b5c-4a19-928b-1b9023991538 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:33.761857 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab41e73-6b5c-4a19-928b-1b9023991538 None admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:33.762613 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab41e73-6b5c-4a19-928b-1b9023991538 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:25:33.762914 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab41e73-6b5c-4a19-928b-1b9023991538 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:25:33.764322 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab41e73-6b5c-4a19-928b-1b9023991538 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:25:33.781525 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 637/1254] 10.222.0.47 () {56 vars in 1143 bytes} [Wed Aug 24 06:25:33 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:25:33.787471 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f982ed-e769-4982-b1c3-02c3a8d68337 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:33.887939 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f982ed-e769-4982-b1c3-02c3a8d68337 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:33.924490 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344be29d-bbf6-4f85-96f9-2c93750e88db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:33.925102 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344be29d-bbf6-4f85-96f9-2c93750e88db None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:33.925427 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344be29d-bbf6-4f85-96f9-2c93750e88db None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:33.925704 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344be29d-bbf6-4f85-96f9-2c93750e88db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:33.984172 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:25:33.996018 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344be29d-bbf6-4f85-96f9-2c93750e88db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:25:33.997492 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344be29d-bbf6-4f85-96f9-2c93750e88db None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:25:34.094985 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 618/1255] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 24 06:25:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 06:25:34.100909 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab41e73-6b5c-4a19-928b-1b9023991538 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:34.202463 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab41e73-6b5c-4a19-928b-1b9023991538 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:34.216141 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:25:34.240004 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed9d138-5a6b-450b-9392-3b1f89ace7b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:34.240004 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed9d138-5a6b-450b-9392-3b1f89ace7b2 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:34.240004 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed9d138-5a6b-450b-9392-3b1f89ace7b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:34.240004 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed9d138-5a6b-450b-9392-3b1f89ace7b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:34.248819 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed9d138-5a6b-450b-9392-3b1f89ace7b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:25:34.250289 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed9d138-5a6b-450b-9392-3b1f89ace7b2 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:25:34.265556 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-7ed9d138-5a6b-450b-9392-3b1f89ace7b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:25:34.265556 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:25:34.265556 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:25:34.265556 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:25:34.265556 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:25:34.265556 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:25:34.265556 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:25:34.265556 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:25:34.265556 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:25:34.265556 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:25:34.265556 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:25:34.265556 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:25:34.265556 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:25:34.265556 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:25:34.265556 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:25:34.265556 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:25:34.269307 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:25:34.269307 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:25:34.269307 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:25:34.269307 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:25:34.269307 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:25:34.269307 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:25:34.269307 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:25:34.269307 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:25:34.269307 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:25:34.269307 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:25:34.269307 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:25:34.269307 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:25:34.269307 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:25:34.269307 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:25:34.269307 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:25:34.269307 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:25:34.269307 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:25:34.270288 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:25:34.270288 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:25:34.270288 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:25:34.270288 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 638/1256] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:25:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:25:34.272357 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344be29d-bbf6-4f85-96f9-2c93750e88db None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:34.357138 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:25:34.369408 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344be29d-bbf6-4f85-96f9-2c93750e88db None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:34.402632 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31713c9-75ed-45d1-a761-b25bf8c370cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:34.403238 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31713c9-75ed-45d1-a761-b25bf8c370cb None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:34.403518 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31713c9-75ed-45d1-a761-b25bf8c370cb None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:34.403792 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31713c9-75ed-45d1-a761-b25bf8c370cb None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:34.404431 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31713c9-75ed-45d1-a761-b25bf8c370cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:25:34.404736 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31713c9-75ed-45d1-a761-b25bf8c370cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:25:34.406228 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31713c9-75ed-45d1-a761-b25bf8c370cb None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:25:34.418597 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 619/1257] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 24 06:25:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:25:34.424951 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed9d138-5a6b-450b-9392-3b1f89ace7b2 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:34.522405 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed9d138-5a6b-450b-9392-3b1f89ace7b2 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:34.556036 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c1abbb-d50b-4758-9e89-5ed30dbe361f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:34.556649 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c1abbb-d50b-4758-9e89-5ed30dbe361f None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:34.556927 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c1abbb-d50b-4758-9e89-5ed30dbe361f None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:34.557218 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c1abbb-d50b-4758-9e89-5ed30dbe361f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:34.563747 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c1abbb-d50b-4758-9e89-5ed30dbe361f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:25:34.565323 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c1abbb-d50b-4758-9e89-5ed30dbe361f None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:25:34.576693 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-49c1abbb-d50b-4758-9e89-5ed30dbe361f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:25:34.576693 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:25:34.576693 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:25:34.576693 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:25:34.576693 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:25:34.576693 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:25:34.576693 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:25:34.576693 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:25:34.576693 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:25:34.576693 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:25:34.576693 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:25:34.576693 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:25:34.576693 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 06:25:34.576693 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 06:25:34.576693 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 06:25:34.577847 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 06:25:34.577847 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:25:34.577847 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:25:34.577847 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:25:34.577847 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:25:34.577847 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:25:34.577847 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:25:34.577847 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:25:34.577847 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:25:34.577847 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:25:34.577847 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:25:34.577847 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:25:34.577847 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:25:34.577847 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:25:34.577847 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:25:34.577847 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:25:34.577847 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:25:34.578926 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 06:25:34.578926 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 06:25:34.578926 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:25:34.578926 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:25:34.578926 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:25:34.578926 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:25:34.578926 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:25:34.578926 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:25:34.578926 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 639/1258] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 24 06:25:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:25:34.583926 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31713c9-75ed-45d1-a761-b25bf8c370cb None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:34.684003 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31713c9-75ed-45d1-a761-b25bf8c370cb None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:34.716857 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6239b22e-0a36-4289-96f3-f8e38db210b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:34.717493 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6239b22e-0a36-4289-96f3-f8e38db210b3 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:34.717775 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6239b22e-0a36-4289-96f3-f8e38db210b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:34.718047 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6239b22e-0a36-4289-96f3-f8e38db210b3 None admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:34.718733 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6239b22e-0a36-4289-96f3-f8e38db210b3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:25:34.719038 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6239b22e-0a36-4289-96f3-f8e38db210b3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:25:34.720519 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6239b22e-0a36-4289-96f3-f8e38db210b3 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:25:34.732958 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 620/1259] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Aug 24 06:25:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:25:34.739792 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c1abbb-d50b-4758-9e89-5ed30dbe361f None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:34.765561 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:25:34.840552 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c1abbb-d50b-4758-9e89-5ed30dbe361f None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:34.874906 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45810357-11ad-4ad8-87ef-27e6a8ea8ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:34.875507 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45810357-11ad-4ad8-87ef-27e6a8ea8ee2 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:34.875787 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45810357-11ad-4ad8-87ef-27e6a8ea8ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:34.876059 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45810357-11ad-4ad8-87ef-27e6a8ea8ee2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:34.876787 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45810357-11ad-4ad8-87ef-27e6a8ea8ee2 None admin] RBAC: Adding query filter params (role.id=486144ad1bcd417e86acdbc1d14327bf, scope.project.id=c02e49a89cdd40d885bfa816f06c58a1, user.id=306188d42dbd4b5b84082ab3842eaf9a) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:25:34.877124 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45810357-11ad-4ad8-87ef-27e6a8ea8ee2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:25:34.878652 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45810357-11ad-4ad8-87ef-27e6a8ea8ee2 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:25:34.884179 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 640/1260] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 24 06:25:34 2022] GET /identity/v3/role_assignments?user.id=306188d42dbd4b5b84082ab3842eaf9a&scope.project.id=c02e49a89cdd40d885bfa816f06c58a1&role.id=486144ad1bcd417e86acdbc1d14327bf => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:25:36.757541 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a99eb7-a98b-4f9a-a3d5-45ea980ee37d None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:36.758065 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a99eb7-a98b-4f9a-a3d5-45ea980ee37d None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:36.758065 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a99eb7-a98b-4f9a-a3d5-45ea980ee37d None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:36.759417 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 621/1261] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:25:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:25:36.764388 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76d10fe-e788-403f-a123-0c30268a4ea3 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:36.764805 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76d10fe-e788-403f-a123-0c30268a4ea3 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:36.764929 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76d10fe-e788-403f-a123-0c30268a4ea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:36.783724 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-e76d10fe-e788-403f-a123-0c30268a4ea3 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:25:36.784898 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-e76d10fe-e788-403f-a123-0c30268a4ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:25:36.849810 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 641/1262] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:25:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:25:37.115590 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a99eb7-a98b-4f9a-a3d5-45ea980ee37d None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:37.126447 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:25:37.216861 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a99eb7-a98b-4f9a-a3d5-45ea980ee37d None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:37.250058 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa61b9e2-5ff3-4d08-b59f-55ed0a6f6750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:37.250782 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa61b9e2-5ff3-4d08-b59f-55ed0a6f6750 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:37.251054 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa61b9e2-5ff3-4d08-b59f-55ed0a6f6750 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:37.251327 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa61b9e2-5ff3-4d08-b59f-55ed0a6f6750 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:37.262302 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa61b9e2-5ff3-4d08-b59f-55ed0a6f6750 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:25:37.266925 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-fa61b9e2-5ff3-4d08-b59f-55ed0a6f6750 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:25:37.266925 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:25:37.266925 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:25:37.266925 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:25:37.266925 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:25:37.266925 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:25:37.266925 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:25:37.266925 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:25:37.266925 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:25:37.266925 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:25:37.266925 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:25:37.266925 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:25:37.266925 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:25:37.266925 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:25:37.266925 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:25:37.266925 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:25:37.268374 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:25:37.268374 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:25:37.268374 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:25:37.268374 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:25:37.268374 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:25:37.268374 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:25:37.268374 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:25:37.268374 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:25:37.268374 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:25:37.268374 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:25:37.268374 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:25:37.268374 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:25:37.268374 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:25:37.268374 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:25:37.268374 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:25:37.268374 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:25:37.276770 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:25:37.276770 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:25:37.276770 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:25:37.276770 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:25:37.276770 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:25:37.276770 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:25:37.276770 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:25:37.276770 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:25:37.276770 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:25:37.276770 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:25:37.276770 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:25:37.276770 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:25:37.276770 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 622/1263] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Aug 24 06:25:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:25:37.277570 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76d10fe-e788-403f-a123-0c30268a4ea3 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:37.293381 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:25:37.374543 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76d10fe-e788-403f-a123-0c30268a4ea3 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:37.409278 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94c4fbe-53df-4506-9826-992b94c4feaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:37.409871 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94c4fbe-53df-4506-9826-992b94c4feaa None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:37.410148 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94c4fbe-53df-4506-9826-992b94c4feaa None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:37.410419 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94c4fbe-53df-4506-9826-992b94c4feaa None admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:37.411119 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94c4fbe-53df-4506-9826-992b94c4feaa None admin] RBAC: Adding query filter params (name=admin) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:25:37.411414 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94c4fbe-53df-4506-9826-992b94c4feaa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:25:37.412767 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94c4fbe-53df-4506-9826-992b94c4feaa None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:25:37.420536 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 642/1264] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 24 06:25:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:25:37.427150 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa61b9e2-5ff3-4d08-b59f-55ed0a6f6750 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:37.491443 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:25:37.524205 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa61b9e2-5ff3-4d08-b59f-55ed0a6f6750 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:37.558148 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50673ee9-167c-40fa-8f45-6f272a8a592f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:37.558777 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50673ee9-167c-40fa-8f45-6f272a8a592f None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:37.559059 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50673ee9-167c-40fa-8f45-6f272a8a592f None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:37.559333 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50673ee9-167c-40fa-8f45-6f272a8a592f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:37.569143 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50673ee9-167c-40fa-8f45-6f272a8a592f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:25:37.570609 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50673ee9-167c-40fa-8f45-6f272a8a592f None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:25:37.585656 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-50673ee9-167c-40fa-8f45-6f272a8a592f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:25:37.585656 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:25:37.585656 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:25:37.585656 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:25:37.585656 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:25:37.585656 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:25:37.585656 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:25:37.585656 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:25:37.585656 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:25:37.585656 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:25:37.585656 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:25:37.585656 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:25:37.585656 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:25:37.585656 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:25:37.585656 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:25:37.585656 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:25:37.587005 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:25:37.587005 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:25:37.587005 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:25:37.587005 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:25:37.587005 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:25:37.587005 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:25:37.587005 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:25:37.587005 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:25:37.587005 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:25:37.587005 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:25:37.587005 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:25:37.587005 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:25:37.587005 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:25:37.587005 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:25:37.587005 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:25:37.587005 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:25:37.587005 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:25:37.588151 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:25:37.588151 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:25:37.588151 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:25:37.588151 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 623/1265] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:25:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:25:37.592572 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94c4fbe-53df-4506-9826-992b94c4feaa None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:37.675965 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:25:37.694571 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94c4fbe-53df-4506-9826-992b94c4feaa None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:37.727459 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a1201e-dfa9-4697-82c6-a96a8247ff61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:37.728066 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a1201e-dfa9-4697-82c6-a96a8247ff61 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:37.728350 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a1201e-dfa9-4697-82c6-a96a8247ff61 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:37.728625 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a1201e-dfa9-4697-82c6-a96a8247ff61 None admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:37.729290 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a1201e-dfa9-4697-82c6-a96a8247ff61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:25:37.729598 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a1201e-dfa9-4697-82c6-a96a8247ff61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:25:37.730994 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a1201e-dfa9-4697-82c6-a96a8247ff61 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:25:37.742775 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 643/1266] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 24 06:25:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:25:37.748527 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50673ee9-167c-40fa-8f45-6f272a8a592f None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:37.848988 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50673ee9-167c-40fa-8f45-6f272a8a592f None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:37.868270 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:25:37.882564 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bcadc2-c197-4912-a1aa-336be4812e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:37.883223 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bcadc2-c197-4912-a1aa-336be4812e38 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:37.883634 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bcadc2-c197-4912-a1aa-336be4812e38 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:37.883905 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bcadc2-c197-4912-a1aa-336be4812e38 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:37.891406 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bcadc2-c197-4912-a1aa-336be4812e38 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:25:37.893373 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bcadc2-c197-4912-a1aa-336be4812e38 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:25:37.904437 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-f7bcadc2-c197-4912-a1aa-336be4812e38 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 06:25:37.904437 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:25:37.904437 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:25:37.904437 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:25:37.904437 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:25:37.904437 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:25:37.904437 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:25:37.904437 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:25:37.904437 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:25:37.904437 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:25:37.904437 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:25:37.904437 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:25:37.904437 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 06:25:37.904437 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 06:25:37.904437 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 06:25:37.904437 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 06:25:37.905404 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:25:37.905404 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:25:37.905404 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 06:25:37.905404 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:25:37.905404 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 06:25:37.905404 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:25:37.905404 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:25:37.905404 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:25:37.905404 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:25:37.905404 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:25:37.905404 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:25:37.905404 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:25:37.905404 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:25:37.905404 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:25:37.905404 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:25:37.905404 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:25:37.905404 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:25:37.905404 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:25:37.906401 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:25:37.906401 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:25:37.906401 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 06:25:37.906401 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 06:25:37.906401 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 06:25:37.906401 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 06:25:37.906401 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 06:25:37.906401 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 06:25:37.906401 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 06:25:37.906401 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:25:37.906401 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 624/1267] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:25:37 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:25:37.912252 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a1201e-dfa9-4697-82c6-a96a8247ff61 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:38.011346 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a1201e-dfa9-4697-82c6-a96a8247ff61 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:38.045240 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d193b0-7da5-4b7a-8ac6-e67dd59960d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:38.045843 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d193b0-7da5-4b7a-8ac6-e67dd59960d3 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:38.046127 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d193b0-7da5-4b7a-8ac6-e67dd59960d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:38.046402 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d193b0-7da5-4b7a-8ac6-e67dd59960d3 None admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:38.047124 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d193b0-7da5-4b7a-8ac6-e67dd59960d3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:25:38.047427 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d193b0-7da5-4b7a-8ac6-e67dd59960d3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:25:38.049163 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d193b0-7da5-4b7a-8ac6-e67dd59960d3 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:25:38.058023 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:25:38.065817 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 644/1268] 10.222.0.47 () {56 vars in 1143 bytes} [Wed Aug 24 06:25:37 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:25:38.071554 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bcadc2-c197-4912-a1aa-336be4812e38 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:38.171139 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bcadc2-c197-4912-a1aa-336be4812e38 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:38.206367 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bab720-d812-4f6a-824d-fa96d106546b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:38.209992 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bab720-d812-4f6a-824d-fa96d106546b None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:38.209992 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bab720-d812-4f6a-824d-fa96d106546b None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:38.209992 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bab720-d812-4f6a-824d-fa96d106546b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:38.217923 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bab720-d812-4f6a-824d-fa96d106546b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:25:38.219413 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bab720-d812-4f6a-824d-fa96d106546b None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:25:38.234032 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-89bab720-d812-4f6a-824d-fa96d106546b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:25:38.234032 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:25:38.234032 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:25:38.234032 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:25:38.234032 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:25:38.234032 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:25:38.234032 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:25:38.234032 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:25:38.234032 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:25:38.234032 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:25:38.234032 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:25:38.234032 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:25:38.234032 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:25:38.234032 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:25:38.234032 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:25:38.234032 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:25:38.235152 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:25:38.235152 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:25:38.235152 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:25:38.235152 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:25:38.235152 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:25:38.235152 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:25:38.235152 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:25:38.235152 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:25:38.235152 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:25:38.235152 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:25:38.235152 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:25:38.235152 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:25:38.235152 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:25:38.235152 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:25:38.235152 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:25:38.235152 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:25:38.235152 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:25:38.236279 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:25:38.236279 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:25:38.236279 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:25:38.236279 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 625/1269] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:25:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:25:38.242509 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d193b0-7da5-4b7a-8ac6-e67dd59960d3 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:38.342344 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d193b0-7da5-4b7a-8ac6-e67dd59960d3 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:38.375034 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980b51cc-39ec-44d2-941c-eef0bdb97b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:38.375639 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980b51cc-39ec-44d2-941c-eef0bdb97b48 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:38.375933 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980b51cc-39ec-44d2-941c-eef0bdb97b48 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:38.376209 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980b51cc-39ec-44d2-941c-eef0bdb97b48 None admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:38.377368 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980b51cc-39ec-44d2-941c-eef0bdb97b48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:25:38.377668 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980b51cc-39ec-44d2-941c-eef0bdb97b48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:25:38.379091 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980b51cc-39ec-44d2-941c-eef0bdb97b48 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:25:38.390352 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 645/1270] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 24 06:25:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:25:38.396898 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bab720-d812-4f6a-824d-fa96d106546b None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:38.427263 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:25:38.493439 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bab720-d812-4f6a-824d-fa96d106546b None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:38.529256 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19ae7b9-5baf-4106-aebe-cf141c865f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:38.529886 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19ae7b9-5baf-4106-aebe-cf141c865f6e None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:38.530165 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19ae7b9-5baf-4106-aebe-cf141c865f6e None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:38.530434 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19ae7b9-5baf-4106-aebe-cf141c865f6e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:38.536979 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19ae7b9-5baf-4106-aebe-cf141c865f6e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:25:38.538430 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19ae7b9-5baf-4106-aebe-cf141c865f6e None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:25:38.549893 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-b19ae7b9-5baf-4106-aebe-cf141c865f6e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:25:38.549893 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:25:38.549893 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:25:38.549893 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:25:38.549893 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:25:38.549893 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:25:38.549893 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:25:38.549893 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:25:38.549893 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:25:38.549893 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:25:38.549893 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:25:38.549893 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:25:38.549893 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 06:25:38.549893 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 06:25:38.549893 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 06:25:38.551049 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 06:25:38.551049 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:25:38.551049 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:25:38.551049 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:25:38.551049 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:25:38.551049 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:25:38.551049 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:25:38.551049 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:25:38.551049 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:25:38.551049 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:25:38.551049 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:25:38.551049 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:25:38.551049 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:25:38.551049 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:25:38.551049 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:25:38.551049 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:25:38.551049 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:25:38.552214 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 06:25:38.552214 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 06:25:38.552214 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:25:38.552214 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:25:38.552214 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:25:38.552214 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:25:38.552214 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:25:38.552214 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:25:38.552214 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 626/1271] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 24 06:25:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:25:38.556810 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980b51cc-39ec-44d2-941c-eef0bdb97b48 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:38.618248 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:25:38.659414 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980b51cc-39ec-44d2-941c-eef0bdb97b48 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:38.692995 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1023b2-fd6e-4a03-b7c1-09509318b921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:38.693644 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1023b2-fd6e-4a03-b7c1-09509318b921 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:38.693947 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1023b2-fd6e-4a03-b7c1-09509318b921 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:38.694226 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1023b2-fd6e-4a03-b7c1-09509318b921 None admin] PATH_INFO: `/v3/projects` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:38.694942 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1023b2-fd6e-4a03-b7c1-09509318b921 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:25:38.695243 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1023b2-fd6e-4a03-b7c1-09509318b921 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:25:38.696673 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1023b2-fd6e-4a03-b7c1-09509318b921 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:25:38.709159 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 646/1272] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Aug 24 06:25:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:25:38.715130 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19ae7b9-5baf-4106-aebe-cf141c865f6e None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:38.806137 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:25:38.815304 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19ae7b9-5baf-4106-aebe-cf141c865f6e None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:38.847409 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c14600-67e8-459f-b13d-7207e5e8a2ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:38.848061 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c14600-67e8-459f-b13d-7207e5e8a2ae None admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:38.848340 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c14600-67e8-459f-b13d-7207e5e8a2ae None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:38.848613 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c14600-67e8-459f-b13d-7207e5e8a2ae None admin] PATH_INFO: `/v3/projects/c02e49a89cdd40d885bfa816f06c58a1/users/306188d42dbd4b5b84082ab3842eaf9a/roles/486144ad1bcd417e86acdbc1d14327bf` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:38.850549 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c14600-67e8-459f-b13d-7207e5e8a2ae None admin] RBAC: Authorizing `identity:create_grant(project_id=c02e49a89cdd40d885bfa816f06c58a1, user_id=306188d42dbd4b5b84082ab3842eaf9a, role_id=486144ad1bcd417e86acdbc1d14327bf)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:25:38.851964 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c14600-67e8-459f-b13d-7207e5e8a2ae None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:25:38.948790 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 627/1273] 10.222.0.47 () {58 vars in 1439 bytes} [Wed Aug 24 06:25:38 2022] PUT /identity/v3/projects/c02e49a89cdd40d885bfa816f06c58a1/users/306188d42dbd4b5b84082ab3842eaf9a/roles/486144ad1bcd417e86acdbc1d14327bf => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:25:40.853408 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1348d6ee-efb2-48c3-80f5-9984de236ffb None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:40.853987 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1348d6ee-efb2-48c3-80f5-9984de236ffb None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:40.853987 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1348d6ee-efb2-48c3-80f5-9984de236ffb None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:40.855295 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 647/1274] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:25:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:25:40.860736 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f2ae7a-8714-44b9-a393-11f221363b74 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:40.861821 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f2ae7a-8714-44b9-a393-11f221363b74 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:40.861821 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f2ae7a-8714-44b9-a393-11f221363b74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:40.880528 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-63f2ae7a-8714-44b9-a393-11f221363b74 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:25:40.881666 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-63f2ae7a-8714-44b9-a393-11f221363b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:25:40.947216 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 628/1275] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:25:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:25:41.216651 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1348d6ee-efb2-48c3-80f5-9984de236ffb None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:41.320386 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1348d6ee-efb2-48c3-80f5-9984de236ffb None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:41.337153 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:25:41.353869 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae679711-0b20-4a0b-aebc-6a8ac9b4bd08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:41.354487 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae679711-0b20-4a0b-aebc-6a8ac9b4bd08 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:41.354764 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae679711-0b20-4a0b-aebc-6a8ac9b4bd08 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:41.355030 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae679711-0b20-4a0b-aebc-6a8ac9b4bd08 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:41.365516 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae679711-0b20-4a0b-aebc-6a8ac9b4bd08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 06:25:41.370111 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-ae679711-0b20-4a0b-aebc-6a8ac9b4bd08 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:25:41.370111 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:25:41.370111 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:25:41.370111 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:25:41.370111 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:25:41.370111 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:25:41.370111 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:25:41.370111 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:25:41.370111 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:25:41.370111 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:25:41.370111 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:25:41.370111 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:25:41.370111 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 06:25:41.370111 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 06:25:41.370111 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 06:25:41.370111 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 06:25:41.371206 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 06:25:41.371206 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 06:25:41.371206 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 06:25:41.371206 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 06:25:41.371206 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:25:41.371206 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:25:41.371206 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:25:41.371206 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:25:41.371206 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:25:41.371206 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:25:41.371206 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:25:41.371206 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:25:41.371206 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:25:41.371206 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:25:41.371206 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:25:41.371206 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:25:41.372214 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:25:41.372214 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:25:41.372214 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:25:41.372214 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:25:41.372214 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 06:25:41.372214 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 06:25:41.372214 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 06:25:41.372214 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 06:25:41.372214 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 06:25:41.372214 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 06:25:41.372214 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 06:25:41.372214 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:25:41.372214 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 648/1276] 10.222.0.47 () {56 vars in 1096 bytes} [Wed Aug 24 06:25:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:25:41.377436 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f2ae7a-8714-44b9-a393-11f221363b74 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:41.457420 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:25:41.475055 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f2ae7a-8714-44b9-a393-11f221363b74 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:41.509749 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70453c0b-4262-46b4-ada3-99ab66cab9bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:41.510351 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70453c0b-4262-46b4-ada3-99ab66cab9bd None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:41.510664 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70453c0b-4262-46b4-ada3-99ab66cab9bd None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:41.510939 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70453c0b-4262-46b4-ada3-99ab66cab9bd None admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:41.511673 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70453c0b-4262-46b4-ada3-99ab66cab9bd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:25:41.511976 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70453c0b-4262-46b4-ada3-99ab66cab9bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:25:41.513359 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70453c0b-4262-46b4-ada3-99ab66cab9bd None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:25:41.521260 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 629/1277] 10.222.0.47 () {56 vars in 1099 bytes} [Wed Aug 24 06:25:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:25:41.526973 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae679711-0b20-4a0b-aebc-6a8ac9b4bd08 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:41.629331 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae679711-0b20-4a0b-aebc-6a8ac9b4bd08 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:41.667295 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09870bf-628e-445b-b2a0-b5e23185a40b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:41.667959 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09870bf-628e-445b-b2a0-b5e23185a40b None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:41.668237 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09870bf-628e-445b-b2a0-b5e23185a40b None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:41.668510 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09870bf-628e-445b-b2a0-b5e23185a40b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:41.678390 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09870bf-628e-445b-b2a0-b5e23185a40b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:25:41.679950 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09870bf-628e-445b-b2a0-b5e23185a40b None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:25:41.694894 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-f09870bf-628e-445b-b2a0-b5e23185a40b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:25:41.694894 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:25:41.694894 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:25:41.694894 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:25:41.694894 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:25:41.694894 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:25:41.694894 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:25:41.694894 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:25:41.694894 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:25:41.694894 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:25:41.694894 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:25:41.694894 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:25:41.694894 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:25:41.694894 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:25:41.694894 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:25:41.694894 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:25:41.699401 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:25:41.699401 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:25:41.699401 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:25:41.699401 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:25:41.699401 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:25:41.699401 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:25:41.699401 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:25:41.699401 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:25:41.699401 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:25:41.699401 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:25:41.699401 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:25:41.699401 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:25:41.699401 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:25:41.699401 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:25:41.699401 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:25:41.699401 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:25:41.699401 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:25:41.700392 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:25:41.700392 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:25:41.700392 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:25:41.700392 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 649/1278] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:25:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:25:41.702038 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70453c0b-4262-46b4-ada3-99ab66cab9bd None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:41.802373 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70453c0b-4262-46b4-ada3-99ab66cab9bd None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:41.835183 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d18b36-8baa-4831-b24e-a1bf158bcb2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:41.835781 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d18b36-8baa-4831-b24e-a1bf158bcb2b None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:41.836060 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d18b36-8baa-4831-b24e-a1bf158bcb2b None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:41.836334 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d18b36-8baa-4831-b24e-a1bf158bcb2b None admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:41.836968 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d18b36-8baa-4831-b24e-a1bf158bcb2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:25:41.837302 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d18b36-8baa-4831-b24e-a1bf158bcb2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:25:41.838698 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d18b36-8baa-4831-b24e-a1bf158bcb2b None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:25:41.850825 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 630/1279] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 24 06:25:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:25:41.856530 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09870bf-628e-445b-b2a0-b5e23185a40b None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:41.889150 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:25:41.959313 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09870bf-628e-445b-b2a0-b5e23185a40b None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:41.992272 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044fa51b-d632-4ec5-8d17-0479d79281e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:41.992896 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044fa51b-d632-4ec5-8d17-0479d79281e4 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:41.993206 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044fa51b-d632-4ec5-8d17-0479d79281e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:41.993480 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044fa51b-d632-4ec5-8d17-0479d79281e4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:42.000633 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044fa51b-d632-4ec5-8d17-0479d79281e4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:25:42.003362 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044fa51b-d632-4ec5-8d17-0479d79281e4 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:25:42.014192 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-044fa51b-d632-4ec5-8d17-0479d79281e4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 06:25:42.014192 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:25:42.014192 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:25:42.014192 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:25:42.014192 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:25:42.014192 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:25:42.014192 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:25:42.014192 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:25:42.014192 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:25:42.014192 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:25:42.014192 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:25:42.014192 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:25:42.014192 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 06:25:42.014192 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 06:25:42.014192 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 06:25:42.014192 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 06:25:42.015300 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:25:42.015300 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:25:42.015300 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 06:25:42.015300 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:25:42.015300 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 06:25:42.015300 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 06:25:42.015300 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:25:42.015300 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:25:42.015300 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:25:42.015300 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:25:42.015300 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:25:42.015300 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:25:42.015300 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:25:42.015300 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:25:42.015300 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:25:42.015300 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:25:42.015300 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:25:42.015300 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:25:42.016659 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:25:42.016659 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:25:42.016659 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 06:25:42.016659 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 06:25:42.016659 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 06:25:42.016659 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 06:25:42.016659 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 06:25:42.016659 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 06:25:42.016659 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 06:25:42.016659 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:25:42.016659 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 650/1280] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:25:41 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:25:42.021744 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d18b36-8baa-4831-b24e-a1bf158bcb2b None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:42.025629 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:25:42.119613 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d18b36-8baa-4831-b24e-a1bf158bcb2b None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:42.154071 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f786630-ea59-4163-995f-4a3cafaae08a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:42.154883 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f786630-ea59-4163-995f-4a3cafaae08a None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:42.155928 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f786630-ea59-4163-995f-4a3cafaae08a None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:42.156260 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f786630-ea59-4163-995f-4a3cafaae08a None admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:42.156982 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f786630-ea59-4163-995f-4a3cafaae08a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:25:42.157310 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f786630-ea59-4163-995f-4a3cafaae08a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:25:42.158729 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f786630-ea59-4163-995f-4a3cafaae08a None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:25:42.175780 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 631/1281] 10.222.0.47 () {56 vars in 1143 bytes} [Wed Aug 24 06:25:42 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:25:42.181698 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044fa51b-d632-4ec5-8d17-0479d79281e4 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:42.271986 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:25:42.281082 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044fa51b-d632-4ec5-8d17-0479d79281e4 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:42.314989 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db123ea4-3b99-4aed-a2ac-23706c2ef7e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:42.315592 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db123ea4-3b99-4aed-a2ac-23706c2ef7e4 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:42.315867 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db123ea4-3b99-4aed-a2ac-23706c2ef7e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:42.316135 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db123ea4-3b99-4aed-a2ac-23706c2ef7e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:42.383761 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db123ea4-3b99-4aed-a2ac-23706c2ef7e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:25:42.385127 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db123ea4-3b99-4aed-a2ac-23706c2ef7e4 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:25:42.471116 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:25:42.481124 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 651/1282] 10.222.0.47 () {58 vars in 1282 bytes} [Wed Aug 24 06:25:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 06:25:42.487827 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f786630-ea59-4163-995f-4a3cafaae08a None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:42.551710 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:25:42.590424 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f786630-ea59-4163-995f-4a3cafaae08a None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:42.624247 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e7fa0b-f8f2-46aa-a7e8-4123fda8b375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:42.624973 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e7fa0b-f8f2-46aa-a7e8-4123fda8b375 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:42.625436 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e7fa0b-f8f2-46aa-a7e8-4123fda8b375 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:42.625704 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e7fa0b-f8f2-46aa-a7e8-4123fda8b375 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:42.636861 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e7fa0b-f8f2-46aa-a7e8-4123fda8b375 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:25:42.638273 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e7fa0b-f8f2-46aa-a7e8-4123fda8b375 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:25:42.653884 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-31e7fa0b-f8f2-46aa-a7e8-4123fda8b375 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:25:42.653884 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:25:42.653884 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:25:42.653884 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:25:42.653884 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:25:42.653884 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:25:42.653884 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:25:42.653884 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:25:42.653884 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:25:42.653884 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:25:42.653884 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:25:42.653884 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:25:42.653884 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 06:25:42.653884 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 06:25:42.653884 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 06:25:42.653884 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 06:25:42.655040 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:25:42.655040 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:25:42.655040 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:25:42.655040 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:25:42.655040 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:25:42.655040 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:25:42.655040 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:25:42.655040 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:25:42.655040 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:25:42.655040 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:25:42.655040 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:25:42.655040 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:25:42.655040 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:25:42.655040 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:25:42.655040 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:25:42.655040 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:25:42.655040 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 06:25:42.655932 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 06:25:42.655932 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 06:25:42.655932 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:25:42.655932 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 632/1283] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:25:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 06:25:42.661327 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db123ea4-3b99-4aed-a2ac-23706c2ef7e4 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:42.761350 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db123ea4-3b99-4aed-a2ac-23706c2ef7e4 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:42.795150 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6320a8-5aec-4a4b-b86c-f10456c5fb74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:42.795757 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6320a8-5aec-4a4b-b86c-f10456c5fb74 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:42.796037 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6320a8-5aec-4a4b-b86c-f10456c5fb74 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:42.796345 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6320a8-5aec-4a4b-b86c-f10456c5fb74 None admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:42.796985 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6320a8-5aec-4a4b-b86c-f10456c5fb74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:25:42.797319 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6320a8-5aec-4a4b-b86c-f10456c5fb74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:25:42.799649 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6320a8-5aec-4a4b-b86c-f10456c5fb74 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:25:42.810700 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 652/1284] 10.222.0.47 () {56 vars in 1109 bytes} [Wed Aug 24 06:25:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:25:42.817032 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e7fa0b-f8f2-46aa-a7e8-4123fda8b375 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:42.914585 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e7fa0b-f8f2-46aa-a7e8-4123fda8b375 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:42.928366 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:25:42.947177 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95caffe4-eca2-48fb-8956-b3d695dbb465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:42.947796 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95caffe4-eca2-48fb-8956-b3d695dbb465 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:42.948070 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95caffe4-eca2-48fb-8956-b3d695dbb465 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:42.948339 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95caffe4-eca2-48fb-8956-b3d695dbb465 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:42.955009 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95caffe4-eca2-48fb-8956-b3d695dbb465 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:25:42.956502 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95caffe4-eca2-48fb-8956-b3d695dbb465 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:25:42.968397 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application [None req-95caffe4-eca2-48fb-8956-b3d695dbb465 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:25:42.968397 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:25:42.968397 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:25:42.968397 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:25:42.968397 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:25:42.968397 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:25:42.968397 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:25:42.968397 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:25:42.968397 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:25:42.968397 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:25:42.968397 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:25:42.968397 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:25:42.968397 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 06:25:42.968397 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 06:25:42.968397 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 06:25:42.971131 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 06:25:42.971131 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:25:42.971131 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:25:42.971131 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:25:42.971131 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:25:42.971131 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:25:42.971131 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:25:42.971131 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:25:42.971131 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application with Lock( Aug 24 06:25:42.971131 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:25:42.971131 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:25:42.971131 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:25:42.971131 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:25:42.971131 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:25:42.971131 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:25:42.971131 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:25:42.971131 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:25:42.972212 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 06:25:42.972212 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 06:25:42.972212 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 06:25:42.972212 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 06:25:42.972212 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 06:25:42.972212 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 06:25:42.972212 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 06:25:42.972212 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: ERROR keystone.server.flask.application Aug 24 06:25:42.972212 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 633/1285] 10.222.0.47 () {56 vars in 1111 bytes} [Wed Aug 24 06:25:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:25:42.976127 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6320a8-5aec-4a4b-b86c-f10456c5fb74 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:43.026824 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:25:43.077652 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6320a8-5aec-4a4b-b86c-f10456c5fb74 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:43.116562 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eedea6-b1e3-4b27-8d36-72f8d5f7af94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:43.117171 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eedea6-b1e3-4b27-8d36-72f8d5f7af94 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:43.117472 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eedea6-b1e3-4b27-8d36-72f8d5f7af94 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:43.117753 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eedea6-b1e3-4b27-8d36-72f8d5f7af94 None admin] PATH_INFO: `/v3/projects` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:43.118441 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0eedea6-b1e3-4b27-8d36-72f8d5f7af94 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:25:43.118768 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0eedea6-b1e3-4b27-8d36-72f8d5f7af94 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:25:43.120199 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0eedea6-b1e3-4b27-8d36-72f8d5f7af94 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:25:43.132836 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 653/1286] 10.222.0.47 () {56 vars in 1148 bytes} [Wed Aug 24 06:25:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:25:43.139041 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95caffe4-eca2-48fb-8956-b3d695dbb465 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:43.236351 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95caffe4-eca2-48fb-8956-b3d695dbb465 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:43.269366 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dce551f-d8fc-4c1e-b094-6799e2f4fcef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:43.270130 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dce551f-d8fc-4c1e-b094-6799e2f4fcef None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:43.270408 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dce551f-d8fc-4c1e-b094-6799e2f4fcef None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:43.270681 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dce551f-d8fc-4c1e-b094-6799e2f4fcef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:43.271405 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dce551f-d8fc-4c1e-b094-6799e2f4fcef None admin] RBAC: Adding query filter params (role.id=486144ad1bcd417e86acdbc1d14327bf, scope.project.id=c02e49a89cdd40d885bfa816f06c58a1, user.id=306188d42dbd4b5b84082ab3842eaf9a) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:25:43.271713 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dce551f-d8fc-4c1e-b094-6799e2f4fcef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:25:43.273131 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dce551f-d8fc-4c1e-b094-6799e2f4fcef None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:25:43.278794 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 634/1287] 10.222.0.47 () {56 vars in 1374 bytes} [Wed Aug 24 06:25:43 2022] GET /identity/v3/role_assignments?user.id=306188d42dbd4b5b84082ab3842eaf9a&scope.project.id=c02e49a89cdd40d885bfa816f06c58a1&role.id=486144ad1bcd417e86acdbc1d14327bf => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:25:45.211876 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d294b30-97b3-4c2e-8c55-5570241cd40a None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:45.212426 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d294b30-97b3-4c2e-8c55-5570241cd40a None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:45.212530 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d294b30-97b3-4c2e-8c55-5570241cd40a None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:45.213823 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 654/1288] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:25:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:25:45.218993 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f89728-2359-4068-a353-00c0a08a171a None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:45.219386 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f89728-2359-4068-a353-00c0a08a171a None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:45.224001 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f89728-2359-4068-a353-00c0a08a171a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:45.238472 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-85f89728-2359-4068-a353-00c0a08a171a None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:25:45.239652 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-85f89728-2359-4068-a353-00c0a08a171a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:25:45.268185 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:25:45.304814 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 635/1289] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:25:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:25:45.577123 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d294b30-97b3-4c2e-8c55-5570241cd40a None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:45.685677 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d294b30-97b3-4c2e-8c55-5570241cd40a None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:45.697529 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:25:45.721410 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564b0f1a-0274-454a-a8f1-c6f7841ab1a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:45.722040 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564b0f1a-0274-454a-a8f1-c6f7841ab1a8 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:45.722331 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564b0f1a-0274-454a-a8f1-c6f7841ab1a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:45.722607 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564b0f1a-0274-454a-a8f1-c6f7841ab1a8 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:45.723195 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564b0f1a-0274-454a-a8f1-c6f7841ab1a8 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:25:45.724545 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564b0f1a-0274-454a-a8f1-c6f7841ab1a8 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:25:45.733143 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-564b0f1a-0274-454a-a8f1-c6f7841ab1a8 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 06:25:45.733143 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:25:45.733143 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 06:25:45.733143 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 06:25:45.733143 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 06:25:45.733143 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 06:25:45.733143 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 06:25:45.733143 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:25:45.733143 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 06:25:45.733143 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:25:45.733143 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 06:25:45.733143 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:25:45.733143 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:25:45.733143 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:25:45.733143 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:25:45.733143 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:25:45.733143 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:25:45.734445 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:25:45.734445 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:25:45.734445 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:25:45.734445 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:25:45.734445 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:25:45.734445 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:25:45.734445 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 06:25:45.734445 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 06:25:45.734445 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 06:25:45.734445 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 06:25:45.734445 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:25:45.734445 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:25:45.734445 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:25:45.734445 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:25:45.734445 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:25:45.734445 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:25:45.735336 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:25:45.735336 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:25:45.735336 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:25:45.735336 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:25:45.735336 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:25:45.735336 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:25:45.735336 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:25:45.735336 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:25:45.735336 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:25:45.735336 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:25:45.735336 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 06:25:45.735336 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:25:45.735336 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 06:25:45.735336 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:25:45.735336 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 655/1290] 10.222.0.47 () {56 vars in 1117 bytes} [Wed Aug 24 06:25:45 2022] GET /identity/v3/services/placement => generated 90 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:25:45.740693 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f89728-2359-4068-a353-00c0a08a171a None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:45.841921 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f89728-2359-4068-a353-00c0a08a171a None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:45.877202 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9b5eff-f8be-4119-8634-ffe2bb6ecdce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:45.877815 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9b5eff-f8be-4119-8634-ffe2bb6ecdce None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:45.878096 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9b5eff-f8be-4119-8634-ffe2bb6ecdce None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:45.878374 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9b5eff-f8be-4119-8634-ffe2bb6ecdce None admin] PATH_INFO: `/v3/services` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:45.879022 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9b5eff-f8be-4119-8634-ffe2bb6ecdce None admin] RBAC: Adding query filter params (name=placement) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:25:45.879320 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9b5eff-f8be-4119-8634-ffe2bb6ecdce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:25:45.880933 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9b5eff-f8be-4119-8634-ffe2bb6ecdce None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:25:45.885716 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 636/1291] 10.222.0.47 () {56 vars in 1116 bytes} [Wed Aug 24 06:25:45 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:25:45.891555 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564b0f1a-0274-454a-a8f1-c6f7841ab1a8 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:45.995296 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564b0f1a-0274-454a-a8f1-c6f7841ab1a8 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:46.033538 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1b0634-7589-4fa5-b8e0-b48777bdb3d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:46.034142 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1b0634-7589-4fa5-b8e0-b48777bdb3d0 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:46.034452 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1b0634-7589-4fa5-b8e0-b48777bdb3d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:46.034831 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1b0634-7589-4fa5-b8e0-b48777bdb3d0 None admin] PATH_INFO: `/v3/services` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:46.035579 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1b0634-7589-4fa5-b8e0-b48777bdb3d0 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:25:46.035939 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1b0634-7589-4fa5-b8e0-b48777bdb3d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:25:46.037342 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1b0634-7589-4fa5-b8e0-b48777bdb3d0 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:25:46.041934 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 656/1292] 10.222.0.47 () {56 vars in 1116 bytes} [Wed Aug 24 06:25:45 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:25:47.962406 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90acb6e6-db39-4313-ba17-ecff0fc7a5d6 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:47.962939 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90acb6e6-db39-4313-ba17-ecff0fc7a5d6 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:47.962939 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90acb6e6-db39-4313-ba17-ecff0fc7a5d6 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:47.964215 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 637/1293] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:25:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:25:47.969396 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75fd0f5-7b1e-4d32-a004-2450434221dd None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:47.976398 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75fd0f5-7b1e-4d32-a004-2450434221dd None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:47.976398 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75fd0f5-7b1e-4d32-a004-2450434221dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:47.987990 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-a75fd0f5-7b1e-4d32-a004-2450434221dd None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:25:47.989164 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-a75fd0f5-7b1e-4d32-a004-2450434221dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:25:48.019586 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:25:48.055435 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 657/1294] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:25:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:25:48.325512 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90acb6e6-db39-4313-ba17-ecff0fc7a5d6 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:48.356527 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:25:48.425107 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90acb6e6-db39-4313-ba17-ecff0fc7a5d6 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:48.458718 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0d7b47-52eb-42ee-9e5c-36d1b2aa6875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:48.459431 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d7b47-52eb-42ee-9e5c-36d1b2aa6875 None admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:48.459714 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d7b47-52eb-42ee-9e5c-36d1b2aa6875 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:48.459989 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d7b47-52eb-42ee-9e5c-36d1b2aa6875 None admin] PATH_INFO: `/v3/services` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:48.460636 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0d7b47-52eb-42ee-9e5c-36d1b2aa6875 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:25:48.462020 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0d7b47-52eb-42ee-9e5c-36d1b2aa6875 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:25:48.540037 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 638/1295] 10.222.0.47 () {60 vars in 1141 bytes} [Wed Aug 24 06:25:48 2022] POST /identity/v3/services => generated 246 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:25:50.486898 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42f1f0c-6ee0-4091-a912-dcb76cc256c8 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:50.487524 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42f1f0c-6ee0-4091-a912-dcb76cc256c8 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:50.487524 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42f1f0c-6ee0-4091-a912-dcb76cc256c8 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:50.488941 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 658/1296] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:25:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:25:50.494768 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbf4abf-affa-458f-9bb4-305265bf1b29 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:50.495207 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbf4abf-affa-458f-9bb4-305265bf1b29 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:50.495301 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbf4abf-affa-458f-9bb4-305265bf1b29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:50.513723 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-0fbf4abf-affa-458f-9bb4-305265bf1b29 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:25:50.514912 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-0fbf4abf-affa-458f-9bb4-305265bf1b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:25:50.586744 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 639/1297] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:25:50 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:25:50.859500 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42f1f0c-6ee0-4091-a912-dcb76cc256c8 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:50.962242 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42f1f0c-6ee0-4091-a912-dcb76cc256c8 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:50.997866 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1463f409-e97e-4dd6-b18f-e2da90921c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:50.998504 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1463f409-e97e-4dd6-b18f-e2da90921c82 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:50.998788 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1463f409-e97e-4dd6-b18f-e2da90921c82 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:50.999067 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1463f409-e97e-4dd6-b18f-e2da90921c82 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:50.999651 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1463f409-e97e-4dd6-b18f-e2da90921c82 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:25:51.001693 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1463f409-e97e-4dd6-b18f-e2da90921c82 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:25:51.010675 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-1463f409-e97e-4dd6-b18f-e2da90921c82 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 06:25:51.010675 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:25:51.010675 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 06:25:51.010675 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 06:25:51.010675 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 06:25:51.010675 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 06:25:51.010675 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 06:25:51.010675 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:25:51.010675 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 06:25:51.010675 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:25:51.010675 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 06:25:51.010675 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:25:51.010675 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:25:51.010675 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:25:51.010675 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:25:51.010675 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:25:51.010675 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:25:51.011927 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:25:51.011927 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:25:51.011927 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:25:51.011927 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:25:51.011927 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:25:51.011927 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:25:51.011927 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 06:25:51.011927 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 06:25:51.011927 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 06:25:51.011927 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 06:25:51.011927 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:25:51.011927 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:25:51.011927 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:25:51.011927 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:25:51.011927 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:25:51.011927 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:25:51.012996 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:25:51.012996 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:25:51.012996 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:25:51.012996 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:25:51.012996 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:25:51.012996 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:25:51.012996 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:25:51.012996 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:25:51.012996 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:25:51.012996 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:25:51.012996 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 06:25:51.012996 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:25:51.012996 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 06:25:51.012996 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:25:51.012996 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 659/1298] 10.222.0.47 () {56 vars in 1117 bytes} [Wed Aug 24 06:25:50 2022] GET /identity/v3/services/placement => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:25:51.021680 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbf4abf-affa-458f-9bb4-305265bf1b29 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:51.036422 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:25:51.118858 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbf4abf-affa-458f-9bb4-305265bf1b29 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:51.153682 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24e5820-84ed-4335-bd29-620582d3ac73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:51.154285 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24e5820-84ed-4335-bd29-620582d3ac73 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:51.154565 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24e5820-84ed-4335-bd29-620582d3ac73 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:51.154837 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24e5820-84ed-4335-bd29-620582d3ac73 None admin] PATH_INFO: `/v3/services` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:51.155463 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24e5820-84ed-4335-bd29-620582d3ac73 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:25:51.155765 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24e5820-84ed-4335-bd29-620582d3ac73 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:25:51.157170 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24e5820-84ed-4335-bd29-620582d3ac73 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:25:51.162883 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 640/1299] 10.222.0.47 () {56 vars in 1116 bytes} [Wed Aug 24 06:25:51 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:25:51.168866 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1463f409-e97e-4dd6-b18f-e2da90921c82 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:51.200845 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:25:51.271281 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1463f409-e97e-4dd6-b18f-e2da90921c82 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:51.306736 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20a1044-8f90-45ef-b397-326d1b0ed7ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:51.307343 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20a1044-8f90-45ef-b397-326d1b0ed7ee None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:51.307638 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20a1044-8f90-45ef-b397-326d1b0ed7ee None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:51.307915 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20a1044-8f90-45ef-b397-326d1b0ed7ee None admin] PATH_INFO: `/v3/endpoints` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:51.308641 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20a1044-8f90-45ef-b397-326d1b0ed7ee None admin] RBAC: Adding query filter params (interface=public, service_id=e06a8e70ed1f422e837228edb5ad1eca, region_id=RegionOne) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:25:51.308916 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20a1044-8f90-45ef-b397-326d1b0ed7ee None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:25:51.310307 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20a1044-8f90-45ef-b397-326d1b0ed7ee None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:25:51.316449 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 660/1300] 10.222.0.47 () {56 vars in 1251 bytes} [Wed Aug 24 06:25:51 2022] GET /identity/v3/endpoints?service_id=e06a8e70ed1f422e837228edb5ad1eca&interface=public®ion_id=RegionOne => generated 194 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:25:51.322174 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24e5820-84ed-4335-bd29-620582d3ac73 None admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:51.404420 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:25:51.421742 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24e5820-84ed-4335-bd29-620582d3ac73 None admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:51.455656 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9958e6f-8dec-485e-9af3-b447d008f085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:51.456260 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9958e6f-8dec-485e-9af3-b447d008f085 None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:51.456541 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9958e6f-8dec-485e-9af3-b447d008f085 None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:51.456822 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9958e6f-8dec-485e-9af3-b447d008f085 None admin] PATH_INFO: `/v3/services` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:51.457449 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9958e6f-8dec-485e-9af3-b447d008f085 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:25:51.458796 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9958e6f-8dec-485e-9af3-b447d008f085 None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:25:51.465545 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 641/1301] 10.222.0.47 () {56 vars in 1087 bytes} [Wed Aug 24 06:25:51 2022] GET /identity/v3/services => generated 2191 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:25:53.423098 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8858ad-253e-4f8d-8a58-d6664bc3d47a None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:53.423639 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8858ad-253e-4f8d-8a58-d6664bc3d47a None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:53.423639 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8858ad-253e-4f8d-8a58-d6664bc3d47a None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:53.424969 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 661/1302] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:25:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:25:53.430274 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82500125-3a1a-41a2-ab1e-45c356afbb6d None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:53.430715 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82500125-3a1a-41a2-ab1e-45c356afbb6d None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:53.430818 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82500125-3a1a-41a2-ab1e-45c356afbb6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:53.449321 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-82500125-3a1a-41a2-ab1e-45c356afbb6d None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:25:53.450512 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-82500125-3a1a-41a2-ab1e-45c356afbb6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:25:53.517199 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 642/1303] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:25:53 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 06:25:53.794156 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8858ad-253e-4f8d-8a58-d6664bc3d47a None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:53.890171 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:25:53.897515 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8858ad-253e-4f8d-8a58-d6664bc3d47a None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:53.931813 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc55a08-2575-4e1a-b327-05c267e9a446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:53.932442 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc55a08-2575-4e1a-b327-05c267e9a446 None admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:53.932723 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc55a08-2575-4e1a-b327-05c267e9a446 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:53.939365 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc55a08-2575-4e1a-b327-05c267e9a446 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:53.939365 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc55a08-2575-4e1a-b327-05c267e9a446 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:25:53.939365 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc55a08-2575-4e1a-b327-05c267e9a446 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:25:53.944191 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application [None req-dcc55a08-2575-4e1a-b327-05c267e9a446 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 06:25:53.944191 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:25:53.944191 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 06:25:53.944191 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 06:25:53.944191 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 06:25:53.944191 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 06:25:53.944191 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 06:25:53.944191 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:25:53.944191 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 06:25:53.944191 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:25:53.944191 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 06:25:53.944191 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:25:53.944191 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 06:25:53.944191 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 06:25:53.944191 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 06:25:53.944191 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 06:25:53.944191 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 06:25:53.945517 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 06:25:53.945517 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 06:25:53.945517 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 06:25:53.945517 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 06:25:53.945517 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 06:25:53.945517 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 06:25:53.945517 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 06:25:53.945517 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 06:25:53.945517 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 06:25:53.945517 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 06:25:53.945517 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 06:25:53.945517 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 06:25:53.945517 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 06:25:53.945517 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 06:25:53.945517 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 06:25:53.945517 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 06:25:53.946462 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 06:25:53.946462 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application with Lock( Aug 24 06:25:53.946462 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 06:25:53.946462 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self._enter() Aug 24 06:25:53.946462 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 06:25:53.946462 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 06:25:53.946462 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 06:25:53.946462 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application return self.creator() Aug 24 06:25:53.946462 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 06:25:53.946462 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application created_value = creator( Aug 24 06:25:53.946462 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 06:25:53.946462 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 06:25:53.946462 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 06:25:53.946462 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: ERROR keystone.server.flask.application Aug 24 06:25:53.946462 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 662/1304] 10.222.0.47 () {56 vars in 1117 bytes} [Wed Aug 24 06:25:53 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 06:25:53.951445 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82500125-3a1a-41a2-ab1e-45c356afbb6d None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:54.026996 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:25:54.053065 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82500125-3a1a-41a2-ab1e-45c356afbb6d None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:54.086499 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ee8cff-b3fc-4b87-914b-b6675572c5af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:54.087105 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ee8cff-b3fc-4b87-914b-b6675572c5af None admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:54.087408 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ee8cff-b3fc-4b87-914b-b6675572c5af None admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:54.087689 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ee8cff-b3fc-4b87-914b-b6675572c5af None admin] PATH_INFO: `/v3/services` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:54.088315 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ee8cff-b3fc-4b87-914b-b6675572c5af None admin] RBAC: Adding query filter params (name=placement) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:25:54.088623 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ee8cff-b3fc-4b87-914b-b6675572c5af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:25:54.090000 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ee8cff-b3fc-4b87-914b-b6675572c5af None admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:25:54.095204 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 643/1305] 10.222.0.47 () {56 vars in 1116 bytes} [Wed Aug 24 06:25:53 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:25:54.101000 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc55a08-2575-4e1a-b327-05c267e9a446 None admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:25:54.202026 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc55a08-2575-4e1a-b327-05c267e9a446 None admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:25:54.235375 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5a446f-58a3-4a05-b0fa-fdb119a1ee27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:25:54.235968 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5a446f-58a3-4a05-b0fa-fdb119a1ee27 None admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:25:54.236242 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5a446f-58a3-4a05-b0fa-fdb119a1ee27 None admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:25:54.236510 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5a446f-58a3-4a05-b0fa-fdb119a1ee27 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:25:54.237163 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5a446f-58a3-4a05-b0fa-fdb119a1ee27 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:25:54.238665 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5a446f-58a3-4a05-b0fa-fdb119a1ee27 None admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:25:54.347916 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 663/1306] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:25:54 2022] POST /identity/v3/endpoints => generated 331 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:26:04.130526 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=30 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:26:04.342870 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:32:15.749590 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688acc83-eac4-4d59-a3cb-f3eafbac21ae None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:15.750059 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688acc83-eac4-4d59-a3cb-f3eafbac21ae None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:15.750146 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688acc83-eac4-4d59-a3cb-f3eafbac21ae None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:15.751473 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 644/1307] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:32:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:32:15.760263 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a3ab48-6982-45e6-a46f-78c1541a314f None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:15.761436 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a3ab48-6982-45e6-a46f-78c1541a314f None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:15.761991 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a3ab48-6982-45e6-a46f-78c1541a314f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:15.850763 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-b0a3ab48-6982-45e6-a46f-78c1541a314f None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:32:15.852002 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-b0a3ab48-6982-45e6-a46f-78c1541a314f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:32:15.970701 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 664/1308] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:32:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:32:16.267963 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644b847c-55dc-4fa6-8ad1-7104ff846d5b None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:16.268267 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644b847c-55dc-4fa6-8ad1-7104ff846d5b None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:16.268511 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644b847c-55dc-4fa6-8ad1-7104ff846d5b None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:16.270049 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 645/1309] 10.222.0.47 () {54 vars in 965 bytes} [Wed Aug 24 06:32:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:32:16.276515 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50aefc4-b3bb-4cde-b45e-57d566991dfa None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:16.276932 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50aefc4-b3bb-4cde-b45e-57d566991dfa None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:16.277316 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50aefc4-b3bb-4cde-b45e-57d566991dfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:16.359973 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-a50aefc4-b3bb-4cde-b45e-57d566991dfa None None] MFA Rules not processed for user `f0ba3f03ec15475ab6ebf64994ad742c`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:32:16.361172 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-a50aefc4-b3bb-4cde-b45e-57d566991dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:32:16.383283 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:32:16.440892 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 665/1310] 10.222.0.47 () {58 vars in 1062 bytes} [Wed Aug 24 06:32:16 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:32:16.450070 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644b847c-55dc-4fa6-8ad1-7104ff846d5b None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:32:16.451934 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-644b847c-55dc-4fa6-8ad1-7104ff846d5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:32:16.596118 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644b847c-55dc-4fa6-8ad1-7104ff846d5b None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:32:16.692039 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7509919b-d075-48c7-8a21-78aa0f5991c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:32:16.692570 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7509919b-d075-48c7-8a21-78aa0f5991c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:16.692846 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7509919b-d075-48c7-8a21-78aa0f5991c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:16.693120 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7509919b-d075-48c7-8a21-78aa0f5991c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:16.694778 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-7509919b-d075-48c7-8a21-78aa0f5991c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:32:16.755635 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:32:16.814767 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7509919b-d075-48c7-8a21-78aa0f5991c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:32:16.816241 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7509919b-d075-48c7-8a21-78aa0f5991c0 service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:32:16.835424 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 646/1311] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:32:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:32:19.066988 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e20d91-e2eb-4ff5-a345-c00da6813beb None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:19.067441 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e20d91-e2eb-4ff5-a345-c00da6813beb None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:19.067535 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e20d91-e2eb-4ff5-a345-c00da6813beb None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:19.069482 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 666/1312] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:32:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:32:19.074914 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f2970d-97fe-4f9d-a509-08c69cdb0104 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:19.075343 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f2970d-97fe-4f9d-a509-08c69cdb0104 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:19.075467 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f2970d-97fe-4f9d-a509-08c69cdb0104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:19.160875 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-d8f2970d-97fe-4f9d-a509-08c69cdb0104 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:32:19.162218 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-d8f2970d-97fe-4f9d-a509-08c69cdb0104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:32:19.268237 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 647/1313] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:32:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:32:19.549291 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e20d91-e2eb-4ff5-a345-c00da6813beb None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:32:19.550904 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-90e20d91-e2eb-4ff5-a345-c00da6813beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:32:19.658603 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e20d91-e2eb-4ff5-a345-c00da6813beb None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:32:19.735737 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:32:19.738375 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbc2ced-0245-49e4-ba8e-22bae886768a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:32:19.738985 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbc2ced-0245-49e4-ba8e-22bae886768a service neutron] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:19.739264 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbc2ced-0245-49e4-ba8e-22bae886768a service neutron] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:19.739539 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbc2ced-0245-49e4-ba8e-22bae886768a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:19.741141 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-0fbc2ced-0245-49e4-ba8e-22bae886768a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:32:19.878878 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbc2ced-0245-49e4-ba8e-22bae886768a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:32:19.878878 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbc2ced-0245-49e4-ba8e-22bae886768a service neutron] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:32:19.878878 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 667/1314] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:32:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:32:22.530280 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e77cb69-b2aa-4603-b806-7ee808e95262 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:22.530876 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e77cb69-b2aa-4603-b806-7ee808e95262 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:22.530876 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e77cb69-b2aa-4603-b806-7ee808e95262 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:22.532148 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 648/1315] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:32:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:32:22.537376 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b300642f-5ed6-4fe4-87fb-6a6e62f8613b None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:22.537828 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b300642f-5ed6-4fe4-87fb-6a6e62f8613b None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:22.537935 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b300642f-5ed6-4fe4-87fb-6a6e62f8613b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:22.604504 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-b300642f-5ed6-4fe4-87fb-6a6e62f8613b None None] MFA Rules not processed for user `b751994501284709b7f528880544d3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:32:22.605747 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-b300642f-5ed6-4fe4-87fb-6a6e62f8613b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:32:22.659892 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:32:22.705201 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 668/1316] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:32:22 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:32:23.004028 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e77cb69-b2aa-4603-b806-7ee808e95262 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:32:23.005614 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-8e77cb69-b2aa-4603-b806-7ee808e95262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:32:23.017987 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:32:23.111384 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e77cb69-b2aa-4603-b806-7ee808e95262 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:32:23.183037 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfc2ef9-9b29-46d1-8c01-eadfd61c2f00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:32:23.183644 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc2ef9-9b29-46d1-8c01-eadfd61c2f00 service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:23.183926 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc2ef9-9b29-46d1-8c01-eadfd61c2f00 service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:23.184198 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc2ef9-9b29-46d1-8c01-eadfd61c2f00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:23.185828 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-cbfc2ef9-9b29-46d1-8c01-eadfd61c2f00 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:32:23.247405 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:32:23.293938 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfc2ef9-9b29-46d1-8c01-eadfd61c2f00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:32:23.295323 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfc2ef9-9b29-46d1-8c01-eadfd61c2f00 service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:32:23.313879 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 649/1317] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:32:23 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:32:26.018262 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7945842c-3a1b-48af-9db6-de3e89dc9017 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:26.018794 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7945842c-3a1b-48af-9db6-de3e89dc9017 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:26.018794 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7945842c-3a1b-48af-9db6-de3e89dc9017 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:26.020065 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 669/1318] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:32:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:32:26.025891 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85a89e2-61a9-48bf-8499-c0b15f542ab5 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:26.026313 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85a89e2-61a9-48bf-8499-c0b15f542ab5 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:26.026740 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85a89e2-61a9-48bf-8499-c0b15f542ab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:26.096132 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-e85a89e2-61a9-48bf-8499-c0b15f542ab5 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:32:26.097375 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-e85a89e2-61a9-48bf-8499-c0b15f542ab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:32:26.190335 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 650/1319] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:32:26 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:32:26.472382 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7945842c-3a1b-48af-9db6-de3e89dc9017 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:32:26.474016 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-7945842c-3a1b-48af-9db6-de3e89dc9017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:32:26.579148 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7945842c-3a1b-48af-9db6-de3e89dc9017 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:32:26.654811 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a8ac80-04ec-4659-a9c5-971538c5246d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:32:26.655412 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a8ac80-04ec-4659-a9c5-971538c5246d service neutron] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:26.655700 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a8ac80-04ec-4659-a9c5-971538c5246d service neutron] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:26.655973 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a8ac80-04ec-4659-a9c5-971538c5246d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:26.657627 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-49a8ac80-04ec-4659-a9c5-971538c5246d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:32:26.665958 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:32:26.772224 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a8ac80-04ec-4659-a9c5-971538c5246d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:32:26.773980 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a8ac80-04ec-4659-a9c5-971538c5246d service neutron] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:32:26.793063 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 670/1320] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:32:26 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:32:29.733900 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40dd3cb-01c7-425b-865f-d7a6123f6bbe None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:29.734479 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40dd3cb-01c7-425b-865f-d7a6123f6bbe None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:29.734479 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40dd3cb-01c7-425b-865f-d7a6123f6bbe None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:29.736059 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 651/1321] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:32:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:32:29.741205 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95c5deb-c809-438f-a9f4-31a27e0d2455 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:29.741613 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95c5deb-c809-438f-a9f4-31a27e0d2455 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:29.741782 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95c5deb-c809-438f-a9f4-31a27e0d2455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:29.811147 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-e95c5deb-c809-438f-a9f4-31a27e0d2455 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:32:29.812359 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-e95c5deb-c809-438f-a9f4-31a27e0d2455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:32:29.871366 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:32:29.910771 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 671/1322] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:32:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:32:30.218071 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801dd1d5-5517-46b3-8828-c5f6343a8055 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:30.218447 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801dd1d5-5517-46b3-8828-c5f6343a8055 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:30.218810 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801dd1d5-5517-46b3-8828-c5f6343a8055 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:30.220249 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 652/1323] 10.222.0.47 () {54 vars in 965 bytes} [Wed Aug 24 06:32:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:32:30.226919 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe294efe-995e-43db-b291-605b52f835cd None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:30.227320 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe294efe-995e-43db-b291-605b52f835cd None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:30.227669 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe294efe-995e-43db-b291-605b52f835cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:30.305480 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-fe294efe-995e-43db-b291-605b52f835cd None None] MFA Rules not processed for user `f0ba3f03ec15475ab6ebf64994ad742c`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:32:30.306689 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-fe294efe-995e-43db-b291-605b52f835cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:32:30.386265 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 672/1324] 10.222.0.47 () {58 vars in 1062 bytes} [Wed Aug 24 06:32:30 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:32:30.395228 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801dd1d5-5517-46b3-8828-c5f6343a8055 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:32:30.396746 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-801dd1d5-5517-46b3-8828-c5f6343a8055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:32:30.417654 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:32:30.503660 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801dd1d5-5517-46b3-8828-c5f6343a8055 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:32:30.576943 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5255a407-944c-4491-8192-48100aca1326 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:32:30.577589 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5255a407-944c-4491-8192-48100aca1326 service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:30.577873 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5255a407-944c-4491-8192-48100aca1326 service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:30.578149 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5255a407-944c-4491-8192-48100aca1326 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:30.579770 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-5255a407-944c-4491-8192-48100aca1326 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:32:30.656266 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:32:30.696251 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5255a407-944c-4491-8192-48100aca1326 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:32:30.698010 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5255a407-944c-4491-8192-48100aca1326 service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:32:30.716258 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 653/1325] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:32:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:32:32.941781 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75546c36-be89-493b-8901-86f63e659b85 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:32.942306 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75546c36-be89-493b-8901-86f63e659b85 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:32.942306 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75546c36-be89-493b-8901-86f63e659b85 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:32.943579 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 673/1326] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:32:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:32:32.949195 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d01e4a7-75f4-476a-9c79-4a6116760444 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:32.949664 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d01e4a7-75f4-476a-9c79-4a6116760444 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:32.949946 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d01e4a7-75f4-476a-9c79-4a6116760444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:33.016495 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-9d01e4a7-75f4-476a-9c79-4a6116760444 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:32:33.017737 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-9d01e4a7-75f4-476a-9c79-4a6116760444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:32:33.126913 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 654/1327] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:32:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:32:33.411972 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75546c36-be89-493b-8901-86f63e659b85 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:32:33.413563 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-75546c36-be89-493b-8901-86f63e659b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:32:33.474793 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:32:33.519470 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75546c36-be89-493b-8901-86f63e659b85 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:32:33.594016 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946e98e7-5a00-4fa4-acbe-d931bb14e46e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:32:33.594698 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946e98e7-5a00-4fa4-acbe-d931bb14e46e service neutron] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:33.594976 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946e98e7-5a00-4fa4-acbe-d931bb14e46e service neutron] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:33.595246 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946e98e7-5a00-4fa4-acbe-d931bb14e46e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:33.596836 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-946e98e7-5a00-4fa4-acbe-d931bb14e46e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:32:33.710954 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:32:33.713531 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946e98e7-5a00-4fa4-acbe-d931bb14e46e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:32:33.714999 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946e98e7-5a00-4fa4-acbe-d931bb14e46e service neutron] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:32:33.734336 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 674/1328] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:32:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:32:35.792654 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49a0455-26de-4e3d-8151-c0affa691f3a None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:35.793101 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49a0455-26de-4e3d-8151-c0affa691f3a None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:35.793231 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49a0455-26de-4e3d-8151-c0affa691f3a None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:35.794766 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 655/1329] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:32:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:32:35.800309 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da183e2-aec2-45c1-b31f-74b97eb8a3c1 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:35.800796 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da183e2-aec2-45c1-b31f-74b97eb8a3c1 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:35.800941 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da183e2-aec2-45c1-b31f-74b97eb8a3c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:35.874358 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-1da183e2-aec2-45c1-b31f-74b97eb8a3c1 None None] MFA Rules not processed for user `b751994501284709b7f528880544d3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:32:35.875532 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-1da183e2-aec2-45c1-b31f-74b97eb8a3c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:32:35.979427 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 675/1330] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:32:35 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:32:36.269475 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49a0455-26de-4e3d-8151-c0affa691f3a None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:32:36.271024 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-e49a0455-26de-4e3d-8151-c0affa691f3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:32:36.302690 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:32:36.375865 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49a0455-26de-4e3d-8151-c0affa691f3a None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:32:36.453057 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229f305a-9b35-401e-ae30-6cce6e53e65e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:32:36.453841 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229f305a-9b35-401e-ae30-6cce6e53e65e service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:36.454118 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229f305a-9b35-401e-ae30-6cce6e53e65e service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:36.454385 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229f305a-9b35-401e-ae30-6cce6e53e65e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:36.456044 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-229f305a-9b35-401e-ae30-6cce6e53e65e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:32:36.538020 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:32:36.573363 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229f305a-9b35-401e-ae30-6cce6e53e65e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:32:36.574636 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229f305a-9b35-401e-ae30-6cce6e53e65e service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:32:36.593487 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 656/1331] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:32:36 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:32:39.102042 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b3701d-3065-4c5a-b5ba-92b2bbc57086 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:39.102546 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b3701d-3065-4c5a-b5ba-92b2bbc57086 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:39.102546 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b3701d-3065-4c5a-b5ba-92b2bbc57086 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:39.103861 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 676/1332] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:32:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:32:39.109172 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d7eed-041b-4dc0-9943-bc707a1d8975 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:39.109592 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d7eed-041b-4dc0-9943-bc707a1d8975 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:39.109746 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d7eed-041b-4dc0-9943-bc707a1d8975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:39.176916 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-e53d7eed-041b-4dc0-9943-bc707a1d8975 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:32:39.178163 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-e53d7eed-041b-4dc0-9943-bc707a1d8975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:32:39.273743 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 657/1333] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:32:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:32:39.560912 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b3701d-3065-4c5a-b5ba-92b2bbc57086 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:32:39.562504 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-a7b3701d-3065-4c5a-b5ba-92b2bbc57086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:32:39.634412 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:32:39.668438 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b3701d-3065-4c5a-b5ba-92b2bbc57086 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:32:39.740717 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f889e29-3836-4cd1-959e-0a4b1b347dfe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:32:39.741352 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f889e29-3836-4cd1-959e-0a4b1b347dfe service neutron] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:39.742288 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f889e29-3836-4cd1-959e-0a4b1b347dfe service neutron] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:39.742562 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f889e29-3836-4cd1-959e-0a4b1b347dfe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:39.744160 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-5f889e29-3836-4cd1-959e-0a4b1b347dfe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:32:39.851132 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f889e29-3836-4cd1-959e-0a4b1b347dfe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:32:39.852742 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f889e29-3836-4cd1-959e-0a4b1b347dfe service neutron] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:32:39.862556 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:32:39.871218 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 677/1334] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:32:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:32:41.820345 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8858f07c-caf7-4bbf-81e2-be5cdac4881f None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:41.820882 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8858f07c-caf7-4bbf-81e2-be5cdac4881f None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:41.820882 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8858f07c-caf7-4bbf-81e2-be5cdac4881f None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:41.822272 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 658/1335] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:32:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:32:41.827416 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fab3d4d-d0ec-4656-adc7-82edba926777 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:41.827833 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fab3d4d-d0ec-4656-adc7-82edba926777 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:41.827973 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fab3d4d-d0ec-4656-adc7-82edba926777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:41.894767 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-8fab3d4d-d0ec-4656-adc7-82edba926777 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:32:41.895918 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-8fab3d4d-d0ec-4656-adc7-82edba926777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:32:41.993368 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 678/1336] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:32:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:32:42.274947 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8858f07c-caf7-4bbf-81e2-be5cdac4881f None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:32:42.276548 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-8858f07c-caf7-4bbf-81e2-be5cdac4881f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:32:42.316471 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:32:42.379363 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8858f07c-caf7-4bbf-81e2-be5cdac4881f None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:32:42.455461 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7211568-d475-4ca0-9a5b-626d0fdc951b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:32:42.456072 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7211568-d475-4ca0-9a5b-626d0fdc951b service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:42.456346 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7211568-d475-4ca0-9a5b-626d0fdc951b service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:42.456612 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7211568-d475-4ca0-9a5b-626d0fdc951b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:42.458392 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-e7211568-d475-4ca0-9a5b-626d0fdc951b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:32:42.544434 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:32:42.570345 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7211568-d475-4ca0-9a5b-626d0fdc951b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:32:42.571528 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7211568-d475-4ca0-9a5b-626d0fdc951b service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:32:42.590539 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 659/1337] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:32:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:32:45.151128 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009aadab-ea7d-4ff3-adb5-ac54170bbd3e None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:45.151717 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009aadab-ea7d-4ff3-adb5-ac54170bbd3e None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:45.151717 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009aadab-ea7d-4ff3-adb5-ac54170bbd3e None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:45.153029 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 679/1338] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:32:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:32:45.158324 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048e9a9d-89eb-498a-b400-91e365b870da None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:45.158757 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048e9a9d-89eb-498a-b400-91e365b870da None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:45.158885 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048e9a9d-89eb-498a-b400-91e365b870da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:45.224917 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-048e9a9d-89eb-498a-b400-91e365b870da None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:32:45.226862 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-048e9a9d-89eb-498a-b400-91e365b870da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:32:45.323173 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 660/1339] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:32:45 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:32:45.615440 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009aadab-ea7d-4ff3-adb5-ac54170bbd3e None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:32:45.617775 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-009aadab-ea7d-4ff3-adb5-ac54170bbd3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:32:45.694834 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:32:45.721868 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009aadab-ea7d-4ff3-adb5-ac54170bbd3e None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:32:45.799826 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075bf191-3168-4629-aab4-6c4ac6606e44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:32:45.800424 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075bf191-3168-4629-aab4-6c4ac6606e44 service neutron] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:45.800698 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075bf191-3168-4629-aab4-6c4ac6606e44 service neutron] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:45.800964 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075bf191-3168-4629-aab4-6c4ac6606e44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:45.802629 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-075bf191-3168-4629-aab4-6c4ac6606e44 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:32:45.911108 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075bf191-3168-4629-aab4-6c4ac6606e44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:32:45.912570 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075bf191-3168-4629-aab4-6c4ac6606e44 service neutron] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:32:45.931845 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 680/1340] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:32:45 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:32:50.064505 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7948333-d563-45a4-96f5-1ef6866b65ec None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:50.065038 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7948333-d563-45a4-96f5-1ef6866b65ec None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:50.065162 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7948333-d563-45a4-96f5-1ef6866b65ec None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:50.066449 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 661/1341] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:32:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:32:50.072804 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f437c7-714e-40a8-89c9-8a5190f79a22 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:50.073280 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f437c7-714e-40a8-89c9-8a5190f79a22 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:50.073406 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f437c7-714e-40a8-89c9-8a5190f79a22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:50.078749 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:32:50.141850 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-77f437c7-714e-40a8-89c9-8a5190f79a22 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:32:50.143056 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-77f437c7-714e-40a8-89c9-8a5190f79a22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:32:50.242219 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 681/1342] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:32:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:32:50.526702 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7948333-d563-45a4-96f5-1ef6866b65ec None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:32:50.528288 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-b7948333-d563-45a4-96f5-1ef6866b65ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:32:50.576363 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:32:50.633977 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7948333-d563-45a4-96f5-1ef6866b65ec None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:32:50.708954 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d865005-1107-4191-aa0e-50b727b21def service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:32:50.710112 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d865005-1107-4191-aa0e-50b727b21def service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:50.710396 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d865005-1107-4191-aa0e-50b727b21def service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:50.710667 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d865005-1107-4191-aa0e-50b727b21def service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:50.712439 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-7d865005-1107-4191-aa0e-50b727b21def service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:32:50.814261 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:32:50.833866 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d865005-1107-4191-aa0e-50b727b21def service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:32:50.836072 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d865005-1107-4191-aa0e-50b727b21def service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:32:50.856047 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 662/1343] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:32:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:32:53.680739 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eac979-17b8-4317-ad58-38a13f7101ed None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:53.681397 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eac979-17b8-4317-ad58-38a13f7101ed None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:53.681397 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eac979-17b8-4317-ad58-38a13f7101ed None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:53.682666 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 682/1344] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:32:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:32:53.687835 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8c1583-d71a-4dad-9d67-b58527b968b5 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:53.688246 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8c1583-d71a-4dad-9d67-b58527b968b5 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:53.688462 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8c1583-d71a-4dad-9d67-b58527b968b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:53.755987 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-cc8c1583-d71a-4dad-9d67-b58527b968b5 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:32:53.757835 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-cc8c1583-d71a-4dad-9d67-b58527b968b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:32:53.880056 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 663/1345] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:32:53 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:32:54.162408 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eac979-17b8-4317-ad58-38a13f7101ed None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:32:54.164047 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-61eac979-17b8-4317-ad58-38a13f7101ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:32:54.257699 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:32:54.274819 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eac979-17b8-4317-ad58-38a13f7101ed None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:32:54.351054 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3450f96f-b046-4569-995c-0fc112c58dcb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:32:54.351671 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3450f96f-b046-4569-995c-0fc112c58dcb service neutron] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:54.351954 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3450f96f-b046-4569-995c-0fc112c58dcb service neutron] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:54.352229 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3450f96f-b046-4569-995c-0fc112c58dcb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:54.353911 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-3450f96f-b046-4569-995c-0fc112c58dcb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:32:54.466656 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3450f96f-b046-4569-995c-0fc112c58dcb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:32:54.468107 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3450f96f-b046-4569-995c-0fc112c58dcb service neutron] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:32:54.486990 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 683/1346] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:32:54 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:32:58.690101 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30098ca-2305-4f17-a7fd-e1717fca0341 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:58.690633 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30098ca-2305-4f17-a7fd-e1717fca0341 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:58.690633 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30098ca-2305-4f17-a7fd-e1717fca0341 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:58.694360 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 664/1347] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:32:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:32:58.698148 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b70162b-64ad-4609-bfc6-1bd8d9335e03 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:58.698604 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b70162b-64ad-4609-bfc6-1bd8d9335e03 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:58.698708 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b70162b-64ad-4609-bfc6-1bd8d9335e03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:58.715414 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:32:58.770911 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-0b70162b-64ad-4609-bfc6-1bd8d9335e03 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:32:58.772794 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-0b70162b-64ad-4609-bfc6-1bd8d9335e03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:32:58.875041 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 684/1348] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:32:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:32:59.163128 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30098ca-2305-4f17-a7fd-e1717fca0341 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:32:59.164635 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-d30098ca-2305-4f17-a7fd-e1717fca0341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:32:59.225777 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:32:59.271124 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30098ca-2305-4f17-a7fd-e1717fca0341 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:32:59.346464 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ce2a7-8cc0-4b3c-970d-853e495f0ca4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:32:59.347404 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ce2a7-8cc0-4b3c-970d-853e495f0ca4 service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:32:59.347404 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ce2a7-8cc0-4b3c-970d-853e495f0ca4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:32:59.347633 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ce2a7-8cc0-4b3c-970d-853e495f0ca4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:32:59.349261 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-5d4ce2a7-8cc0-4b3c-970d-853e495f0ca4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:32:59.461594 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:32:59.465115 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4ce2a7-8cc0-4b3c-970d-853e495f0ca4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:32:59.466586 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4ce2a7-8cc0-4b3c-970d-853e495f0ca4 service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:32:59.485145 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 665/1349] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:32:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:33:04.413635 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91634518-8ae1-4d94-a06b-56780834d659 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:04.413635 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91634518-8ae1-4d94-a06b-56780834d659 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:04.413635 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91634518-8ae1-4d94-a06b-56780834d659 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:04.415052 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 685/1350] 10.222.0.47 () {54 vars in 921 bytes} [Wed Aug 24 06:33:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:33:04.420125 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfb3ebd-d5a6-4473-8974-246c958798ec None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:04.420612 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfb3ebd-d5a6-4473-8974-246c958798ec None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:04.422156 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfb3ebd-d5a6-4473-8974-246c958798ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:04.520896 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-dbfb3ebd-d5a6-4473-8974-246c958798ec None None] MFA Rules not processed for user `306188d42dbd4b5b84082ab3842eaf9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:33:04.522995 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-dbfb3ebd-d5a6-4473-8974-246c958798ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:33:04.673207 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 666/1351] 10.222.0.47 () {58 vars in 1018 bytes} [Wed Aug 24 06:33:04 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:33:08.873211 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=59 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:08.953074 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e5d639-f58e-43f4-a93b-8139b12a60f2 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:08.953526 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e5d639-f58e-43f4-a93b-8139b12a60f2 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:08.953738 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e5d639-f58e-43f4-a93b-8139b12a60f2 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:08.955197 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 686/1352] 10.222.0.47 () {54 vars in 921 bytes} [Wed Aug 24 06:33:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:33:08.959900 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a95b2d8-3984-40d2-99dc-76272f9790b5 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:08.960405 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a95b2d8-3984-40d2-99dc-76272f9790b5 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:08.960488 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a95b2d8-3984-40d2-99dc-76272f9790b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:09.015280 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:09.039566 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-8a95b2d8-3984-40d2-99dc-76272f9790b5 None None] MFA Rules not processed for user `306188d42dbd4b5b84082ab3842eaf9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:33:09.041369 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-8a95b2d8-3984-40d2-99dc-76272f9790b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:33:09.156097 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 667/1353] 10.222.0.47 () {58 vars in 1018 bytes} [Wed Aug 24 06:33:08 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:33:10.378934 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811d1bc2-4df6-4fae-b9c8-3875d3a9ad90 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:10.379512 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811d1bc2-4df6-4fae-b9c8-3875d3a9ad90 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:10.379512 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811d1bc2-4df6-4fae-b9c8-3875d3a9ad90 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:10.381938 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 687/1354] 10.222.0.47 () {54 vars in 921 bytes} [Wed Aug 24 06:33:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:33:10.387234 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cdf4bb-abda-46ab-9637-e278e866c158 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:10.387677 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cdf4bb-abda-46ab-9637-e278e866c158 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:10.387970 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cdf4bb-abda-46ab-9637-e278e866c158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:10.465607 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-99cdf4bb-abda-46ab-9637-e278e866c158 None None] MFA Rules not processed for user `306188d42dbd4b5b84082ab3842eaf9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:33:10.466857 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-99cdf4bb-abda-46ab-9637-e278e866c158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:33:10.509585 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:10.575248 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 668/1355] 10.222.0.47 () {58 vars in 1018 bytes} [Wed Aug 24 06:33:10 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:33:13.972279 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceab6f8e-944e-4500-93b2-991a062cc632 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:13.972822 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceab6f8e-944e-4500-93b2-991a062cc632 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:13.973001 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceab6f8e-944e-4500-93b2-991a062cc632 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:13.974701 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 688/1356] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:33:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:33:13.980637 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba67a51-c524-4063-8803-66a0c5e868f4 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:13.981055 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba67a51-c524-4063-8803-66a0c5e868f4 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:13.981175 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba67a51-c524-4063-8803-66a0c5e868f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:14.052415 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-4ba67a51-c524-4063-8803-66a0c5e868f4 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:33:14.053140 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-4ba67a51-c524-4063-8803-66a0c5e868f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:33:14.185897 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 669/1357] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:33:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:33:14.494692 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8eb8ea-3d77-4a63-af68-c3cb2b56c1f2 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:14.495125 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8eb8ea-3d77-4a63-af68-c3cb2b56c1f2 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:14.495610 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8eb8ea-3d77-4a63-af68-c3cb2b56c1f2 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:14.497579 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 689/1358] 10.222.0.47 () {54 vars in 962 bytes} [Wed Aug 24 06:33:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:33:14.504161 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e5cc26-a634-4d65-ad04-195cf136e6d2 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:14.504666 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e5cc26-a634-4d65-ad04-195cf136e6d2 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:14.505381 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e5cc26-a634-4d65-ad04-195cf136e6d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:14.524043 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:14.587111 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-05e5cc26-a634-4d65-ad04-195cf136e6d2 None None] MFA Rules not processed for user `1f4f49c1b08c460f8ba6c451c9c92c2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:33:14.588481 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-05e5cc26-a634-4d65-ad04-195cf136e6d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:33:14.722483 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 670/1359] 10.222.0.47 () {58 vars in 1059 bytes} [Wed Aug 24 06:33:14 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:33:14.730890 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8eb8ea-3d77-4a63-af68-c3cb2b56c1f2 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:14.732478 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-6d8eb8ea-3d77-4a63-af68-c3cb2b56c1f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:33:14.902467 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8eb8ea-3d77-4a63-af68-c3cb2b56c1f2 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:14.981737 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:15.026875 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdb53ff-32a3-495b-b02e-a4ef9b400503 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4f49c1b08c460f8ba6c451c9c92c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:15.027485 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdb53ff-32a3-495b-b02e-a4ef9b400503 service nova] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:15.027758 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdb53ff-32a3-495b-b02e-a4ef9b400503 service nova] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:15.028022 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdb53ff-32a3-495b-b02e-a4ef9b400503 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:15.029734 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-9bdb53ff-32a3-495b-b02e-a4ef9b400503 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:33:15.124916 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdb53ff-32a3-495b-b02e-a4ef9b400503 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:15.126767 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdb53ff-32a3-495b-b02e-a4ef9b400503 service nova] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:15.146437 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 690/1360] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:33:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:33:17.291467 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a601b1-fa5e-407d-b310-2348d3a24025 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:17.291467 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a601b1-fa5e-407d-b310-2348d3a24025 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:17.291467 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a601b1-fa5e-407d-b310-2348d3a24025 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:17.293009 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 671/1361] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:33:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:33:17.297854 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4727538c-6603-43e7-aaeb-24f6f14ccc16 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:17.298232 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4727538c-6603-43e7-aaeb-24f6f14ccc16 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:17.298656 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4727538c-6603-43e7-aaeb-24f6f14ccc16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:17.371647 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-4727538c-6603-43e7-aaeb-24f6f14ccc16 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:33:17.373001 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-4727538c-6603-43e7-aaeb-24f6f14ccc16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:33:17.418319 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:17.481327 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 691/1362] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:33:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:33:17.774926 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a601b1-fa5e-407d-b310-2348d3a24025 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:17.776579 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-a1a601b1-fa5e-407d-b310-2348d3a24025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:33:17.851958 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:17.972894 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a601b1-fa5e-407d-b310-2348d3a24025 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:18.096474 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e98aa5-75b4-429d-9676-1c3d67cfbf1c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4f49c1b08c460f8ba6c451c9c92c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:18.097064 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e98aa5-75b4-429d-9676-1c3d67cfbf1c service nova] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:18.098407 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e98aa5-75b4-429d-9676-1c3d67cfbf1c service nova] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:18.098692 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e98aa5-75b4-429d-9676-1c3d67cfbf1c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:18.100884 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-64e98aa5-75b4-429d-9676-1c3d67cfbf1c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:33:18.125132 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:18.195025 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e98aa5-75b4-429d-9676-1c3d67cfbf1c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:18.196634 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e98aa5-75b4-429d-9676-1c3d67cfbf1c service nova] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:18.216556 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 672/1363] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:33:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:33:20.613268 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b6527e-876e-4060-8ef9-caf5e1d5da59 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:20.613793 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b6527e-876e-4060-8ef9-caf5e1d5da59 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:20.613909 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b6527e-876e-4060-8ef9-caf5e1d5da59 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:20.616137 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 692/1364] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:33:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:33:20.623164 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56454cad-e4e7-4dbe-ae50-c50939b73ec9 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:20.623164 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56454cad-e4e7-4dbe-ae50-c50939b73ec9 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:20.623600 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56454cad-e4e7-4dbe-ae50-c50939b73ec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:20.701170 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-56454cad-e4e7-4dbe-ae50-c50939b73ec9 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:33:20.702436 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-56454cad-e4e7-4dbe-ae50-c50939b73ec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:33:20.805425 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:20.808767 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 673/1365] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:33:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:33:21.104534 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00953315-6c72-49c4-b13b-e04b242f9431 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:21.104819 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00953315-6c72-49c4-b13b-e04b242f9431 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:21.105400 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00953315-6c72-49c4-b13b-e04b242f9431 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:21.106927 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 693/1366] 10.222.0.47 () {54 vars in 962 bytes} [Wed Aug 24 06:33:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:33:21.113311 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e20c902-490c-4471-b723-657f41f8bc55 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:21.118978 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e20c902-490c-4471-b723-657f41f8bc55 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:21.118978 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e20c902-490c-4471-b723-657f41f8bc55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:21.195716 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-2e20c902-490c-4471-b723-657f41f8bc55 None None] MFA Rules not processed for user `1f4f49c1b08c460f8ba6c451c9c92c2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:33:21.196979 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-2e20c902-490c-4471-b723-657f41f8bc55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:33:21.315408 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 674/1367] 10.222.0.47 () {58 vars in 1059 bytes} [Wed Aug 24 06:33:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:33:21.323219 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00953315-6c72-49c4-b13b-e04b242f9431 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:21.324922 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-00953315-6c72-49c4-b13b-e04b242f9431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:33:21.484660 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00953315-6c72-49c4-b13b-e04b242f9431 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:21.507357 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:21.664880 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bea4eb8-fa64-4a79-9de0-bc62bc09f963 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4f49c1b08c460f8ba6c451c9c92c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:21.664880 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bea4eb8-fa64-4a79-9de0-bc62bc09f963 service nova] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:21.664880 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bea4eb8-fa64-4a79-9de0-bc62bc09f963 service nova] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:21.664880 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bea4eb8-fa64-4a79-9de0-bc62bc09f963 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:21.666580 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-1bea4eb8-fa64-4a79-9de0-bc62bc09f963 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:33:21.723988 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bea4eb8-fa64-4a79-9de0-bc62bc09f963 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:21.742662 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bea4eb8-fa64-4a79-9de0-bc62bc09f963 service nova] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:21.745539 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 694/1368] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:33:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:33:22.424629 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4936df32-dd55-458a-8ed8-fa9f81e51988 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:22.425215 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4936df32-dd55-458a-8ed8-fa9f81e51988 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:22.425215 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4936df32-dd55-458a-8ed8-fa9f81e51988 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:22.426785 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 675/1369] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:33:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:33:22.432070 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c113e9f2-056c-49a3-ab87-df5b40d29ba3 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:22.432363 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c113e9f2-056c-49a3-ab87-df5b40d29ba3 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:22.432635 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c113e9f2-056c-49a3-ab87-df5b40d29ba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:22.466379 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:22.504733 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-c113e9f2-056c-49a3-ab87-df5b40d29ba3 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:33:22.506583 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-c113e9f2-056c-49a3-ab87-df5b40d29ba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:33:22.613254 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 695/1370] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:33:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:33:23.219047 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800a035e-6795-4d1e-9a27-14f8916289aa None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:23.220120 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800a035e-6795-4d1e-9a27-14f8916289aa None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:23.220120 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800a035e-6795-4d1e-9a27-14f8916289aa None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:23.222113 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 676/1371] 10.222.0.47 () {54 vars in 964 bytes} [Wed Aug 24 06:33:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:33:23.227115 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacc5a79-aca4-49a4-af46-ed26765fe1e4 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:23.227630 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacc5a79-aca4-49a4-af46-ed26765fe1e4 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:23.227794 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacc5a79-aca4-49a4-af46-ed26765fe1e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:23.306021 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-aacc5a79-aca4-49a4-af46-ed26765fe1e4 None None] MFA Rules not processed for user `36639dce4f97412f89d71367f650b901`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:33:23.307263 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-aacc5a79-aca4-49a4-af46-ed26765fe1e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:33:23.373941 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:23.407857 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 696/1372] 10.222.0.47 () {58 vars in 1061 bytes} [Wed Aug 24 06:33:23 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:33:23.416583 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800a035e-6795-4d1e-9a27-14f8916289aa None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:23.418581 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-800a035e-6795-4d1e-9a27-14f8916289aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:33:23.473114 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:23.530871 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800a035e-6795-4d1e-9a27-14f8916289aa None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:23.612761 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200587d4-e74d-4213-8da1-1655c1944d3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36639dce4f97412f89d71367f650b901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:23.613412 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200587d4-e74d-4213-8da1-1655c1944d3c service cinder] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:23.613722 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200587d4-e74d-4213-8da1-1655c1944d3c service cinder] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:23.614002 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200587d4-e74d-4213-8da1-1655c1944d3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:23.616349 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-200587d4-e74d-4213-8da1-1655c1944d3c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:33:23.718577 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:23.732653 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200587d4-e74d-4213-8da1-1655c1944d3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:23.734098 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200587d4-e74d-4213-8da1-1655c1944d3c service cinder] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:23.752285 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 677/1373] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:33:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:33:23.980702 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff66ee7e-15b1-40f9-bfab-1eb8feeb0b38 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:23.980911 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff66ee7e-15b1-40f9-bfab-1eb8feeb0b38 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:23.981648 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff66ee7e-15b1-40f9-bfab-1eb8feeb0b38 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:23.983945 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 697/1374] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:33:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:33:23.990039 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41348da4-e0ad-40e7-ac57-0cb4c4575acb None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:23.990249 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41348da4-e0ad-40e7-ac57-0cb4c4575acb None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:23.990591 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41348da4-e0ad-40e7-ac57-0cb4c4575acb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:24.087150 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-41348da4-e0ad-40e7-ac57-0cb4c4575acb None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:33:24.089064 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-41348da4-e0ad-40e7-ac57-0cb4c4575acb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:33:24.215496 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 678/1375] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:33:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:33:24.506956 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff66ee7e-15b1-40f9-bfab-1eb8feeb0b38 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:24.508494 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-ff66ee7e-15b1-40f9-bfab-1eb8feeb0b38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:33:24.668045 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff66ee7e-15b1-40f9-bfab-1eb8feeb0b38 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:24.720146 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:24.784300 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8330753b-c050-4066-aa01-72923b6868f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4f49c1b08c460f8ba6c451c9c92c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:24.784904 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8330753b-c050-4066-aa01-72923b6868f2 service nova] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:24.785205 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8330753b-c050-4066-aa01-72923b6868f2 service nova] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:24.785480 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8330753b-c050-4066-aa01-72923b6868f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:24.787685 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-8330753b-c050-4066-aa01-72923b6868f2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:33:24.879578 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8330753b-c050-4066-aa01-72923b6868f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:24.882238 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8330753b-c050-4066-aa01-72923b6868f2 service nova] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:24.900624 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 698/1376] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:33:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:33:26.995481 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4bdd43-66f4-463c-a4ce-3c88c0d1eae5 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:26.996026 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4bdd43-66f4-463c-a4ce-3c88c0d1eae5 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:26.996026 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4bdd43-66f4-463c-a4ce-3c88c0d1eae5 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:26.997719 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 679/1377] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:33:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:33:27.003366 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bfbab6-7d7e-4a3c-8cf2-aa51566e7e0a None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:27.003843 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bfbab6-7d7e-4a3c-8cf2-aa51566e7e0a None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:27.003960 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bfbab6-7d7e-4a3c-8cf2-aa51566e7e0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:27.073220 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-93bfbab6-7d7e-4a3c-8cf2-aa51566e7e0a None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:33:27.074352 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-93bfbab6-7d7e-4a3c-8cf2-aa51566e7e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:33:27.081057 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:27.174709 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 699/1378] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:33:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:33:27.477258 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4bdd43-66f4-463c-a4ce-3c88c0d1eae5 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:27.479871 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-2b4bdd43-66f4-463c-a4ce-3c88c0d1eae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:33:27.564083 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:27.658606 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4bdd43-66f4-463c-a4ce-3c88c0d1eae5 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:27.787469 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4294799f-2325-43db-8f6a-0c3d6833970f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4f49c1b08c460f8ba6c451c9c92c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:27.788092 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4294799f-2325-43db-8f6a-0c3d6833970f service nova] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:27.788374 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4294799f-2325-43db-8f6a-0c3d6833970f service nova] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:27.788653 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4294799f-2325-43db-8f6a-0c3d6833970f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:27.790300 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-4294799f-2325-43db-8f6a-0c3d6833970f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:33:27.809541 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:27.886130 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4294799f-2325-43db-8f6a-0c3d6833970f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:27.887762 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4294799f-2325-43db-8f6a-0c3d6833970f service nova] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:27.906311 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 680/1379] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:33:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:33:30.031745 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8000ff70-622a-44d3-bc58-a6ce0e884d46 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:30.032255 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8000ff70-622a-44d3-bc58-a6ce0e884d46 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:30.032255 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8000ff70-622a-44d3-bc58-a6ce0e884d46 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:30.033868 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 700/1380] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:33:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:33:30.039505 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db13314-10d9-4feb-99c9-70f89adbabc9 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:30.039917 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db13314-10d9-4feb-99c9-70f89adbabc9 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:30.040156 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db13314-10d9-4feb-99c9-70f89adbabc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:30.111177 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-9db13314-10d9-4feb-99c9-70f89adbabc9 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:33:30.112400 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-9db13314-10d9-4feb-99c9-70f89adbabc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:33:30.210628 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:30.217012 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 681/1381] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:33:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:33:30.522129 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8000ff70-622a-44d3-bc58-a6ce0e884d46 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:30.523721 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-8000ff70-622a-44d3-bc58-a6ce0e884d46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:33:30.682577 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:30.694165 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8000ff70-622a-44d3-bc58-a6ce0e884d46 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:30.816749 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8b84f1-803a-4d25-84dd-dc4d0317dc7e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4f49c1b08c460f8ba6c451c9c92c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:30.817840 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8b84f1-803a-4d25-84dd-dc4d0317dc7e service nova] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:30.818143 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8b84f1-803a-4d25-84dd-dc4d0317dc7e service nova] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:30.818375 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8b84f1-803a-4d25-84dd-dc4d0317dc7e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:30.820027 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-bc8b84f1-803a-4d25-84dd-dc4d0317dc7e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:33:30.915805 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8b84f1-803a-4d25-84dd-dc4d0317dc7e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:30.917609 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8b84f1-803a-4d25-84dd-dc4d0317dc7e service nova] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:30.928638 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:30.941259 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 701/1382] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:33:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:33:33.071659 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e67b841-fc9e-4239-8440-c95bc6338467 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:33.072211 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e67b841-fc9e-4239-8440-c95bc6338467 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:33.072211 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e67b841-fc9e-4239-8440-c95bc6338467 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:33.073686 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 682/1383] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:33:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:33:33.079942 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0957d755-b36c-443a-b6d3-83870769fea8 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:33.080414 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0957d755-b36c-443a-b6d3-83870769fea8 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:33.080607 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0957d755-b36c-443a-b6d3-83870769fea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:33.150598 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-0957d755-b36c-443a-b6d3-83870769fea8 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:33:33.151889 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-0957d755-b36c-443a-b6d3-83870769fea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:33:33.262238 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 702/1384] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:33:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:33:33.610740 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e67b841-fc9e-4239-8440-c95bc6338467 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:33.612476 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-9e67b841-fc9e-4239-8440-c95bc6338467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:33:33.791907 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e67b841-fc9e-4239-8440-c95bc6338467 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:33.875164 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:33.927168 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4c76ce-669d-425e-bb01-b14aa2753c80 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4f49c1b08c460f8ba6c451c9c92c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:33.927765 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4c76ce-669d-425e-bb01-b14aa2753c80 service nova] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:33.928078 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4c76ce-669d-425e-bb01-b14aa2753c80 service nova] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:33.928380 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4c76ce-669d-425e-bb01-b14aa2753c80 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:33.930227 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-aa4c76ce-669d-425e-bb01-b14aa2753c80 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:33:34.041339 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4c76ce-669d-425e-bb01-b14aa2753c80 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:34.043007 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4c76ce-669d-425e-bb01-b14aa2753c80 service nova] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:34.063775 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 683/1385] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:33:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:33:36.343429 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe642493-8230-40c8-980b-2681bc12fdae None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:36.344018 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe642493-8230-40c8-980b-2681bc12fdae None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:36.344243 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe642493-8230-40c8-980b-2681bc12fdae None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:36.346463 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 703/1386] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:33:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:33:36.352498 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a965bef-594a-4a2e-878e-ab4b9a172134 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:36.352974 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a965bef-594a-4a2e-878e-ab4b9a172134 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:36.353068 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a965bef-594a-4a2e-878e-ab4b9a172134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:36.429020 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-5a965bef-594a-4a2e-878e-ab4b9a172134 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:33:36.430537 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-5a965bef-594a-4a2e-878e-ab4b9a172134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:33:36.469742 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:36.556081 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 684/1387] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:33:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:33:36.962563 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe642493-8230-40c8-980b-2681bc12fdae None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:36.964107 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-fe642493-8230-40c8-980b-2681bc12fdae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:33:37.048797 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:37.122876 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe642493-8230-40c8-980b-2681bc12fdae None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:37.250168 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deffa3b3-35d6-41cd-b37e-138498024141 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4f49c1b08c460f8ba6c451c9c92c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:37.250667 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deffa3b3-35d6-41cd-b37e-138498024141 service nova] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:37.250956 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deffa3b3-35d6-41cd-b37e-138498024141 service nova] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:37.251237 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deffa3b3-35d6-41cd-b37e-138498024141 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:37.252980 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-deffa3b3-35d6-41cd-b37e-138498024141 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:33:37.297414 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:37.361872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deffa3b3-35d6-41cd-b37e-138498024141 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:37.364254 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deffa3b3-35d6-41cd-b37e-138498024141 service nova] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:37.383973 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 704/1388] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:33:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:33:39.411317 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a367fbb9-66ee-4c2b-bc5a-5b927e5d649b None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:39.411861 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a367fbb9-66ee-4c2b-bc5a-5b927e5d649b None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:39.411861 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a367fbb9-66ee-4c2b-bc5a-5b927e5d649b None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:39.413391 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 685/1389] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:33:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:33:39.418421 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcc8c3c-f250-443e-b2fe-a30d829f4ad9 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:39.418861 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcc8c3c-f250-443e-b2fe-a30d829f4ad9 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:39.418999 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcc8c3c-f250-443e-b2fe-a30d829f4ad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:39.486109 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-5fcc8c3c-f250-443e-b2fe-a30d829f4ad9 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:33:39.487316 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-5fcc8c3c-f250-443e-b2fe-a30d829f4ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:33:39.515512 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf8a605-21c5-4dd2-869f-b5c28d1deccf None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:39.515792 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf8a605-21c5-4dd2-869f-b5c28d1deccf None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:39.516063 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf8a605-21c5-4dd2-869f-b5c28d1deccf None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:39.517461 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 686/1390] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:33:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 06:33:39.522581 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a26bc8d-8a41-4ae8-a893-ca9894b7c25c None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:39.522890 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a26bc8d-8a41-4ae8-a893-ca9894b7c25c None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:39.523159 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a26bc8d-8a41-4ae8-a893-ca9894b7c25c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:39.586985 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 705/1391] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:33:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:33:39.590421 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-7a26bc8d-8a41-4ae8-a893-ca9894b7c25c None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:33:39.591649 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-7a26bc8d-8a41-4ae8-a893-ca9894b7c25c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:33:39.692168 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 687/1392] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:33:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:33:39.889669 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922f9bf2-c4a2-48dc-9beb-d15bb0ad8102 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:39.889951 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922f9bf2-c4a2-48dc-9beb-d15bb0ad8102 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:39.890230 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922f9bf2-c4a2-48dc-9beb-d15bb0ad8102 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:39.891576 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 706/1393] 10.222.0.47 () {54 vars in 967 bytes} [Wed Aug 24 06:33:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:33:39.897464 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98e80c9-4e61-412e-a249-bb75c8a66817 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:39.897873 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98e80c9-4e61-412e-a249-bb75c8a66817 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:39.898179 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98e80c9-4e61-412e-a249-bb75c8a66817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:39.914249 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:39.976182 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-c98e80c9-4e61-412e-a249-bb75c8a66817 None None] MFA Rules not processed for user `31b331b6ddfe46aeae7b52fcbbc27771`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:33:39.977405 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-c98e80c9-4e61-412e-a249-bb75c8a66817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:33:39.982022 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922f9bf2-c4a2-48dc-9beb-d15bb0ad8102 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:39.983520 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-922f9bf2-c4a2-48dc-9beb-d15bb0ad8102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:33:39.997286 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:40.055643 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 688/1394] 10.222.0.47 () {58 vars in 1064 bytes} [Wed Aug 24 06:33:39 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:33:40.063230 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98e80c9-4e61-412e-a249-bb75c8a66817 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:40.071143 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98e80c9-4e61-412e-a249-bb75c8a66817 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:40.138155 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922f9bf2-c4a2-48dc-9beb-d15bb0ad8102 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:40.147142 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded05256-086c-4875-9c8d-0bc73348f0f5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b331b6ddfe46aeae7b52fcbbc27771', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:40.147760 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded05256-086c-4875-9c8d-0bc73348f0f5 service glance] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:40.148039 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded05256-086c-4875-9c8d-0bc73348f0f5 service glance] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:40.148311 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded05256-086c-4875-9c8d-0bc73348f0f5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:40.149966 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-ded05256-086c-4875-9c8d-0bc73348f0f5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:33:40.180798 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:40.231199 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:40.259603 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7610420a-30a8-4be1-8859-635bf3e11a98 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4f49c1b08c460f8ba6c451c9c92c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:40.260407 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7610420a-30a8-4be1-8859-635bf3e11a98 service nova] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:40.260987 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7610420a-30a8-4be1-8859-635bf3e11a98 service nova] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:40.261288 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7610420a-30a8-4be1-8859-635bf3e11a98 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:40.262614 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded05256-086c-4875-9c8d-0bc73348f0f5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:40.262907 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-7610420a-30a8-4be1-8859-635bf3e11a98 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:33:40.264047 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded05256-086c-4875-9c8d-0bc73348f0f5 service glance] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:40.283747 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 689/1395] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:33:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:33:40.358092 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7610420a-30a8-4be1-8859-635bf3e11a98 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:40.359707 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7610420a-30a8-4be1-8859-635bf3e11a98 service nova] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:40.378715 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 707/1396] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:33:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:33:41.151403 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c9fa52-1b7f-4c90-be8f-fa4da76e65f6 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:41.151937 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c9fa52-1b7f-4c90-be8f-fa4da76e65f6 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:41.152021 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c9fa52-1b7f-4c90-be8f-fa4da76e65f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:41.230754 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-b2c9fa52-1b7f-4c90-be8f-fa4da76e65f6 None None] MFA Rules not processed for user `ea70acd744994250a2905b5dd7163dfb`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:33:41.231956 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-b2c9fa52-1b7f-4c90-be8f-fa4da76e65f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:33:41.318593 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:41.324543 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 690/1397] 10.222.0.47 () {58 vars in 1009 bytes} [Wed Aug 24 06:33:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:33:41.371238 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e6c528-b9aa-4853-b21c-5a65888a4590 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:41.371761 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e6c528-b9aa-4853-b21c-5a65888a4590 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:41.371857 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e6c528-b9aa-4853-b21c-5a65888a4590 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:41.373144 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 708/1398] 10.222.0.47 () {54 vars in 943 bytes} [Wed Aug 24 06:33:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:33:41.380737 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b77f7d0-2815-402f-86b6-e2eb228afedc None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:41.381027 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b77f7d0-2815-402f-86b6-e2eb228afedc None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:41.381349 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b77f7d0-2815-402f-86b6-e2eb228afedc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:41.472066 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-2b77f7d0-2815-402f-86b6-e2eb228afedc None None] MFA Rules not processed for user `02f1cbaf2cd6427585f298c9d58c319a`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:33:41.473489 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-2b77f7d0-2815-402f-86b6-e2eb228afedc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:33:41.606456 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 691/1399] 10.222.0.47 () {58 vars in 1040 bytes} [Wed Aug 24 06:33:41 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:33:41.629655 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e6c528-b9aa-4853-b21c-5a65888a4590 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:41.629655 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-c3e6c528-b9aa-4853-b21c-5a65888a4590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:33:41.749676 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:41.794873 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e6c528-b9aa-4853-b21c-5a65888a4590 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:41.914792 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b3dd3b-7891-4f03-8e1e-f0fc162b49ab service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f1cbaf2cd6427585f298c9d58c319a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:41.915403 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b3dd3b-7891-4f03-8e1e-f0fc162b49ab service swift] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:41.915685 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b3dd3b-7891-4f03-8e1e-f0fc162b49ab service swift] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:41.915968 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b3dd3b-7891-4f03-8e1e-f0fc162b49ab service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:41.917657 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-29b3dd3b-7891-4f03-8e1e-f0fc162b49ab service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:33:42.010730 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b3dd3b-7891-4f03-8e1e-f0fc162b49ab service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:42.012267 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b3dd3b-7891-4f03-8e1e-f0fc162b49ab service swift] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:42.016039 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:42.022548 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 709/1400] 10.222.0.47 () {60 vars in 1384 bytes} [Wed Aug 24 06:33:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 06:33:42.452507 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b84e35-2206-46e0-8ef0-f2ae9128f56a None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:42.453168 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b84e35-2206-46e0-8ef0-f2ae9128f56a None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:42.453168 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b84e35-2206-46e0-8ef0-f2ae9128f56a None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:42.454607 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 692/1401] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:33:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:33:42.460706 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d73b05-6abb-4493-8d20-9179f382b271 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:42.461274 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d73b05-6abb-4493-8d20-9179f382b271 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:42.461274 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d73b05-6abb-4493-8d20-9179f382b271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:42.537606 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-d4d73b05-6abb-4493-8d20-9179f382b271 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:33:42.538955 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-d4d73b05-6abb-4493-8d20-9179f382b271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:33:42.668051 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 710/1402] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:33:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:33:42.977986 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b84e35-2206-46e0-8ef0-f2ae9128f56a None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:42.979532 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-30b84e35-2206-46e0-8ef0-f2ae9128f56a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:33:43.054010 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:43.173298 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b84e35-2206-46e0-8ef0-f2ae9128f56a None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:43.335044 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcb770f-6f43-440d-a14b-379cdb21fb27 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4f49c1b08c460f8ba6c451c9c92c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:43.335929 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcb770f-6f43-440d-a14b-379cdb21fb27 service nova] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:43.336222 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcb770f-6f43-440d-a14b-379cdb21fb27 service nova] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:43.336736 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcb770f-6f43-440d-a14b-379cdb21fb27 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:43.338464 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-adcb770f-6f43-440d-a14b-379cdb21fb27 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:33:43.361438 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:43.450947 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcb770f-6f43-440d-a14b-379cdb21fb27 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:43.452718 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcb770f-6f43-440d-a14b-379cdb21fb27 service nova] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:43.472180 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 693/1403] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:33:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:33:45.619416 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615e21a2-dd64-42d8-a23a-0c38b65f2ec6 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:45.620009 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615e21a2-dd64-42d8-a23a-0c38b65f2ec6 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:45.620009 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615e21a2-dd64-42d8-a23a-0c38b65f2ec6 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:45.625572 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 711/1404] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:33:45 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:33:45.626909 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ec2bf4-effd-4662-a9f8-c0a412b17445 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:45.627352 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ec2bf4-effd-4662-a9f8-c0a412b17445 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:45.627459 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ec2bf4-effd-4662-a9f8-c0a412b17445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:45.704975 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-59ec2bf4-effd-4662-a9f8-c0a412b17445 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:33:45.706259 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-59ec2bf4-effd-4662-a9f8-c0a412b17445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:33:45.812961 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:45.825224 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 694/1405] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:33:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:33:46.120147 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615e21a2-dd64-42d8-a23a-0c38b65f2ec6 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:46.121926 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-615e21a2-dd64-42d8-a23a-0c38b65f2ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:33:46.233708 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:46.323863 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615e21a2-dd64-42d8-a23a-0c38b65f2ec6 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:46.463059 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695bdaf8-56db-4796-aa7c-dbb30dd708c0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4f49c1b08c460f8ba6c451c9c92c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:46.463671 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695bdaf8-56db-4796-aa7c-dbb30dd708c0 service nova] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:46.463944 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695bdaf8-56db-4796-aa7c-dbb30dd708c0 service nova] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:46.464214 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695bdaf8-56db-4796-aa7c-dbb30dd708c0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:46.465953 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-695bdaf8-56db-4796-aa7c-dbb30dd708c0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:33:46.510761 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:46.576977 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695bdaf8-56db-4796-aa7c-dbb30dd708c0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:46.578571 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695bdaf8-56db-4796-aa7c-dbb30dd708c0 service nova] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:46.599523 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 712/1406] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:33:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:33:48.859906 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf25edf-1ab0-4edd-9b03-494515465601 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:48.861234 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf25edf-1ab0-4edd-9b03-494515465601 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:48.861761 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf25edf-1ab0-4edd-9b03-494515465601 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:48.863381 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 695/1407] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:33:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:33:48.871797 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0241ce5-f797-485c-a457-c558f9a31daf None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:48.871797 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0241ce5-f797-485c-a457-c558f9a31daf None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:48.871797 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0241ce5-f797-485c-a457-c558f9a31daf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:48.944724 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-d0241ce5-f797-485c-a457-c558f9a31daf None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:33:48.945977 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-d0241ce5-f797-485c-a457-c558f9a31daf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:33:49.054887 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 713/1408] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:33:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:33:49.359905 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf25edf-1ab0-4edd-9b03-494515465601 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:33:49.361468 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-0cf25edf-1ab0-4edd-9b03-494515465601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:33:49.570498 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf25edf-1ab0-4edd-9b03-494515465601 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:33:49.730509 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:49.783653 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a02f8f2-b5cc-4895-a8ac-dda60544023f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4f49c1b08c460f8ba6c451c9c92c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:33:49.784284 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a02f8f2-b5cc-4895-a8ac-dda60544023f service nova] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:33:49.785165 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a02f8f2-b5cc-4895-a8ac-dda60544023f service nova] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:33:49.785525 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a02f8f2-b5cc-4895-a8ac-dda60544023f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:33:49.787219 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-5a02f8f2-b5cc-4895-a8ac-dda60544023f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:33:49.893767 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a02f8f2-b5cc-4895-a8ac-dda60544023f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:33:49.895364 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a02f8f2-b5cc-4895-a8ac-dda60544023f service nova] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:33:49.916260 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 696/1409] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:33:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 557 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:33:59.052835 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=93 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:33:59.913855 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=65 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:34:06.672352 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fc3f4e-c10a-4053-ae10-31131e90eec7 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:06.672922 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fc3f4e-c10a-4053-ae10-31131e90eec7 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:06.672922 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fc3f4e-c10a-4053-ae10-31131e90eec7 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:06.674341 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 714/1410] 10.222.0.47 () {54 vars in 918 bytes} [Wed Aug 24 06:34:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:34:06.679988 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eb7161-41a4-4bde-b5aa-da03f7f352d4 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:06.680485 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eb7161-41a4-4bde-b5aa-da03f7f352d4 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:06.680588 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eb7161-41a4-4bde-b5aa-da03f7f352d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:06.770790 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-f7eb7161-41a4-4bde-b5aa-da03f7f352d4 None None] MFA Rules not processed for user `306188d42dbd4b5b84082ab3842eaf9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:34:06.772045 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-f7eb7161-41a4-4bde-b5aa-da03f7f352d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:34:06.898535 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 697/1411] 10.222.0.47 () {58 vars in 1015 bytes} [Wed Aug 24 06:34:06 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:34:06.920036 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15df23dc-b57c-4f44-83fb-703867225a76 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:06.920534 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15df23dc-b57c-4f44-83fb-703867225a76 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:06.921022 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15df23dc-b57c-4f44-83fb-703867225a76 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:06.923504 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 715/1412] 10.222.0.47 () {54 vars in 918 bytes} [Wed Aug 24 06:34:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:34:06.930550 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f5e75f-6afb-451f-b637-6d33912bcca4 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:06.930839 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f5e75f-6afb-451f-b637-6d33912bcca4 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:06.931122 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f5e75f-6afb-451f-b637-6d33912bcca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:07.019908 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:34:07.029206 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-a1f5e75f-6afb-451f-b637-6d33912bcca4 None None] MFA Rules not processed for user `1f4f49c1b08c460f8ba6c451c9c92c2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:34:07.030468 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-a1f5e75f-6afb-451f-b637-6d33912bcca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:34:07.172500 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 698/1413] 10.222.0.47 () {58 vars in 1015 bytes} [Wed Aug 24 06:34:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:34:09.736512 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f6a9ed-6688-4aee-a824-8919a1daaf6e None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:09.737262 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f6a9ed-6688-4aee-a824-8919a1daaf6e None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:09.737402 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f6a9ed-6688-4aee-a824-8919a1daaf6e None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:09.738886 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 716/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 06:34:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:34:09.745938 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9298f2e7-9c50-41d3-9979-b10db1cce4e5 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:09.746743 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9298f2e7-9c50-41d3-9979-b10db1cce4e5 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:09.746949 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9298f2e7-9c50-41d3-9979-b10db1cce4e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:09.834557 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-9298f2e7-9c50-41d3-9979-b10db1cce4e5 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:34:09.836572 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-9298f2e7-9c50-41d3-9979-b10db1cce4e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:34:09.907948 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:34:09.959234 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 699/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 06:34:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:34:10.257554 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f6a9ed-6688-4aee-a824-8919a1daaf6e None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:34:10.259105 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-76f6a9ed-6688-4aee-a824-8919a1daaf6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:34:10.425898 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f6a9ed-6688-4aee-a824-8919a1daaf6e None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:34:10.527119 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:34:10.594169 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1783cb0c-473e-4626-94cf-8f594fe11931 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4f49c1b08c460f8ba6c451c9c92c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:34:10.595041 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1783cb0c-473e-4626-94cf-8f594fe11931 service nova] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:10.595477 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1783cb0c-473e-4626-94cf-8f594fe11931 service nova] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:10.595888 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1783cb0c-473e-4626-94cf-8f594fe11931 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:10.599206 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-1783cb0c-473e-4626-94cf-8f594fe11931 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:34:10.721774 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1783cb0c-473e-4626-94cf-8f594fe11931 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:34:10.723550 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1783cb0c-473e-4626-94cf-8f594fe11931 service nova] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:34:10.743492 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 717/1416] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:34:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:34:11.795911 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ffc74-5f63-42f6-ac47-aab7829b6992 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:11.796538 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ffc74-5f63-42f6-ac47-aab7829b6992 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:11.796912 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ffc74-5f63-42f6-ac47-aab7829b6992 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:11.799621 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 700/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 06:34:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:34:11.805635 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f3005a-ab73-4afe-b498-9c9fd6ae4cb4 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:11.806441 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f3005a-ab73-4afe-b498-9c9fd6ae4cb4 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:11.806441 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f3005a-ab73-4afe-b498-9c9fd6ae4cb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:11.907634 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-74f3005a-ab73-4afe-b498-9c9fd6ae4cb4 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:34:11.910994 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-74f3005a-ab73-4afe-b498-9c9fd6ae4cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:34:11.950414 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:34:12.047844 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 718/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 06:34:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:34:12.206391 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810ffc74-5f63-42f6-ac47-aab7829b6992 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:34:12.207494 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-810ffc74-5f63-42f6-ac47-aab7829b6992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:34:12.456517 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810ffc74-5f63-42f6-ac47-aab7829b6992 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:34:12.517357 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:34:12.621886 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf526079-f1ba-4e0f-abb1-a65e4e138f11 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4f49c1b08c460f8ba6c451c9c92c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:34:12.622507 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf526079-f1ba-4e0f-abb1-a65e4e138f11 service nova] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:12.622803 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf526079-f1ba-4e0f-abb1-a65e4e138f11 service nova] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:12.623045 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf526079-f1ba-4e0f-abb1-a65e4e138f11 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:12.624951 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-cf526079-f1ba-4e0f-abb1-a65e4e138f11 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:34:12.745992 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf526079-f1ba-4e0f-abb1-a65e4e138f11 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:34:12.747535 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf526079-f1ba-4e0f-abb1-a65e4e138f11 service nova] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:34:12.768563 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 701/1419] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:34:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 564 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:34:13.823224 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5f92bf-b217-4d31-9097-165c8630fc64 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:13.823882 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5f92bf-b217-4d31-9097-165c8630fc64 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:13.823882 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5f92bf-b217-4d31-9097-165c8630fc64 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:13.825146 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 719/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 06:34:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:34:13.831398 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e37bb32-a690-4eac-8631-a6c1c32927d7 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:13.831959 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e37bb32-a690-4eac-8631-a6c1c32927d7 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:13.832205 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e37bb32-a690-4eac-8631-a6c1c32927d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:13.902933 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:34:13.930850 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-1e37bb32-a690-4eac-8631-a6c1c32927d7 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:34:13.932133 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-1e37bb32-a690-4eac-8631-a6c1c32927d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:34:14.049844 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 702/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 06:34:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:34:14.206602 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5f92bf-b217-4d31-9097-165c8630fc64 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:34:14.209443 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-ca5f92bf-b217-4d31-9097-165c8630fc64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:34:14.391919 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5f92bf-b217-4d31-9097-165c8630fc64 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:34:14.406428 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:34:14.529786 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9defdd-2a63-4cea-a11c-d7d5999cb5ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4f49c1b08c460f8ba6c451c9c92c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:34:14.530400 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9defdd-2a63-4cea-a11c-d7d5999cb5ad service nova] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:14.530682 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9defdd-2a63-4cea-a11c-d7d5999cb5ad service nova] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:14.530966 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9defdd-2a63-4cea-a11c-d7d5999cb5ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:14.532829 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-ef9defdd-2a63-4cea-a11c-d7d5999cb5ad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:34:14.652682 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9defdd-2a63-4cea-a11c-d7d5999cb5ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:34:14.654367 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9defdd-2a63-4cea-a11c-d7d5999cb5ad service nova] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:34:14.680068 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 720/1422] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:34:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:34:15.623971 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57a6c04-be36-4c79-88c3-93844d77ec48 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:15.624719 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57a6c04-be36-4c79-88c3-93844d77ec48 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:15.624844 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57a6c04-be36-4c79-88c3-93844d77ec48 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:15.626363 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 703/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 06:34:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:34:15.632465 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353bd57b-f063-4f4e-a94a-dbabf73bbd62 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:15.633055 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353bd57b-f063-4f4e-a94a-dbabf73bbd62 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:15.633529 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353bd57b-f063-4f4e-a94a-dbabf73bbd62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:15.661797 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:34:15.741118 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-353bd57b-f063-4f4e-a94a-dbabf73bbd62 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:34:15.742545 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-353bd57b-f063-4f4e-a94a-dbabf73bbd62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:34:15.868710 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 721/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 06:34:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:34:16.025035 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57a6c04-be36-4c79-88c3-93844d77ec48 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:34:16.027981 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-b57a6c04-be36-4c79-88c3-93844d77ec48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:34:16.162204 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:34:16.206886 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57a6c04-be36-4c79-88c3-93844d77ec48 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:34:16.366559 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933639eb-c290-4aaa-b26c-9a2b92772ef4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4f49c1b08c460f8ba6c451c9c92c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:34:16.367173 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933639eb-c290-4aaa-b26c-9a2b92772ef4 service nova] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:16.367450 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933639eb-c290-4aaa-b26c-9a2b92772ef4 service nova] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:16.367922 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933639eb-c290-4aaa-b26c-9a2b92772ef4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:16.369616 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-933639eb-c290-4aaa-b26c-9a2b92772ef4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:34:16.457051 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:34:16.477122 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933639eb-c290-4aaa-b26c-9a2b92772ef4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:34:16.478755 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933639eb-c290-4aaa-b26c-9a2b92772ef4 service nova] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:34:16.499775 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 704/1425] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:34:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:34:17.511067 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5304c9d5-6984-46af-83dc-2942c95f2375 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:17.511789 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5304c9d5-6984-46af-83dc-2942c95f2375 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:17.511789 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5304c9d5-6984-46af-83dc-2942c95f2375 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:17.513000 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 722/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 06:34:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:34:17.519438 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9507c20e-6086-4715-bffb-a9abd566c080 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:17.520190 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9507c20e-6086-4715-bffb-a9abd566c080 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:17.520190 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9507c20e-6086-4715-bffb-a9abd566c080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:17.626874 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-9507c20e-6086-4715-bffb-a9abd566c080 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:34:17.628553 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-9507c20e-6086-4715-bffb-a9abd566c080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:34:17.772146 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 705/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 06:34:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:34:17.937884 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5304c9d5-6984-46af-83dc-2942c95f2375 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:34:17.939988 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-5304c9d5-6984-46af-83dc-2942c95f2375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:34:18.050250 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:34:18.127182 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5304c9d5-6984-46af-83dc-2942c95f2375 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:34:18.267045 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93765af2-f1fb-4ac8-9629-d18fc56e6bf8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4f49c1b08c460f8ba6c451c9c92c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:34:18.268082 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93765af2-f1fb-4ac8-9629-d18fc56e6bf8 service nova] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:18.268407 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93765af2-f1fb-4ac8-9629-d18fc56e6bf8 service nova] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:18.268892 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93765af2-f1fb-4ac8-9629-d18fc56e6bf8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:18.270781 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-93765af2-f1fb-4ac8-9629-d18fc56e6bf8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:34:18.331368 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:34:18.382275 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93765af2-f1fb-4ac8-9629-d18fc56e6bf8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:34:18.385816 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93765af2-f1fb-4ac8-9629-d18fc56e6bf8 service nova] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:34:18.415669 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 723/1428] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:34:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:34:19.430816 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0454af-bf67-4fa8-93b3-a5cf054a767d None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:19.431401 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0454af-bf67-4fa8-93b3-a5cf054a767d None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:19.431401 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0454af-bf67-4fa8-93b3-a5cf054a767d None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:19.432910 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 706/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 06:34:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:34:19.440950 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07ef4ba-419b-4a9a-8e7a-26a6b0b5d12e None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:19.441563 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07ef4ba-419b-4a9a-8e7a-26a6b0b5d12e None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:19.441853 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07ef4ba-419b-4a9a-8e7a-26a6b0b5d12e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:19.560429 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-e07ef4ba-419b-4a9a-8e7a-26a6b0b5d12e None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:34:19.561698 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-e07ef4ba-419b-4a9a-8e7a-26a6b0b5d12e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:34:19.685765 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 724/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 06:34:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:34:19.844482 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0454af-bf67-4fa8-93b3-a5cf054a767d None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:34:19.846311 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-ab0454af-bf67-4fa8-93b3-a5cf054a767d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:34:19.908468 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:34:20.050802 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0454af-bf67-4fa8-93b3-a5cf054a767d None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:34:20.224337 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592b474c-dc56-4180-8890-fd8172abe85d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4f49c1b08c460f8ba6c451c9c92c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:34:20.224869 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592b474c-dc56-4180-8890-fd8172abe85d service nova] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:20.225144 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592b474c-dc56-4180-8890-fd8172abe85d service nova] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:20.225451 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592b474c-dc56-4180-8890-fd8172abe85d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:20.227144 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-592b474c-dc56-4180-8890-fd8172abe85d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:34:20.235213 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:34:20.355167 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592b474c-dc56-4180-8890-fd8172abe85d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:34:20.356932 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592b474c-dc56-4180-8890-fd8172abe85d service nova] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:34:20.381817 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 707/1431] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:34:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 538 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:34:21.375930 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fade04c-755b-4183-ad6e-5baca3b8e6cd None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:21.375930 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fade04c-755b-4183-ad6e-5baca3b8e6cd None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:21.375930 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fade04c-755b-4183-ad6e-5baca3b8e6cd None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:21.375930 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 725/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 06:34:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:34:21.380017 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d892ffad-17a3-4044-8f06-bd103d10890e None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:21.380750 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d892ffad-17a3-4044-8f06-bd103d10890e None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:21.380750 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d892ffad-17a3-4044-8f06-bd103d10890e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:21.488377 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-d892ffad-17a3-4044-8f06-bd103d10890e None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:34:21.490343 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-d892ffad-17a3-4044-8f06-bd103d10890e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:34:21.584173 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:34:21.616985 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 708/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 06:34:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:34:21.775793 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fade04c-755b-4183-ad6e-5baca3b8e6cd None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:34:21.778231 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-6fade04c-755b-4183-ad6e-5baca3b8e6cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:34:21.801488 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:34:21.968666 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fade04c-755b-4183-ad6e-5baca3b8e6cd None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:34:22.079476 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:34:22.131184 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4115bd7e-c14f-4dfc-ad65-5aceaaca2df0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4f49c1b08c460f8ba6c451c9c92c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:34:22.132133 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4115bd7e-c14f-4dfc-ad65-5aceaaca2df0 service nova] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:22.132559 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4115bd7e-c14f-4dfc-ad65-5aceaaca2df0 service nova] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:22.132998 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4115bd7e-c14f-4dfc-ad65-5aceaaca2df0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:22.135832 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-4115bd7e-c14f-4dfc-ad65-5aceaaca2df0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:34:22.279407 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4115bd7e-c14f-4dfc-ad65-5aceaaca2df0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:34:22.281053 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4115bd7e-c14f-4dfc-ad65-5aceaaca2df0 service nova] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:34:22.305711 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 726/1434] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:34:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 531 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:34:23.243845 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f31cd86-db80-4030-930d-369d86595b1c None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:23.244509 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f31cd86-db80-4030-930d-369d86595b1c None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:23.244509 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f31cd86-db80-4030-930d-369d86595b1c None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:23.245797 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 709/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 06:34:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 06:34:23.251754 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa2d4d6-335e-463d-881b-413ae92bc6bd None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:23.252382 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa2d4d6-335e-463d-881b-413ae92bc6bd None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:23.253926 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa2d4d6-335e-463d-881b-413ae92bc6bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:23.347906 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-dfa2d4d6-335e-463d-881b-413ae92bc6bd None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:34:23.349169 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-dfa2d4d6-335e-463d-881b-413ae92bc6bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:34:23.399981 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:34:23.469694 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 727/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 06:34:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:34:23.625377 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f31cd86-db80-4030-930d-369d86595b1c None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:34:23.627574 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-7f31cd86-db80-4030-930d-369d86595b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:34:23.791432 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f31cd86-db80-4030-930d-369d86595b1c None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:34:23.849513 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:34:23.915896 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722d00f2-e770-49f3-b706-0b44f859fa36 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4f49c1b08c460f8ba6c451c9c92c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:34:23.916605 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722d00f2-e770-49f3-b706-0b44f859fa36 service nova] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:23.916885 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722d00f2-e770-49f3-b706-0b44f859fa36 service nova] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:23.917162 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722d00f2-e770-49f3-b706-0b44f859fa36 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:23.918823 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-722d00f2-e770-49f3-b706-0b44f859fa36 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:34:24.012451 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722d00f2-e770-49f3-b706-0b44f859fa36 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:34:24.014115 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722d00f2-e770-49f3-b706-0b44f859fa36 service nova] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:34:24.034432 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 710/1437] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:34:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:34:24.942119 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf41ecb-39a2-4b98-8623-d40134b59da6 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:24.942666 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf41ecb-39a2-4b98-8623-d40134b59da6 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:24.942666 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf41ecb-39a2-4b98-8623-d40134b59da6 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:24.944045 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 728/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 06:34:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:34:24.949489 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1feb01-8f53-47a9-9f13-2a15b49031ca None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:24.949978 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1feb01-8f53-47a9-9f13-2a15b49031ca None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:24.950083 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1feb01-8f53-47a9-9f13-2a15b49031ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:25.034757 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-bf1feb01-8f53-47a9-9f13-2a15b49031ca None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:34:25.035992 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-bf1feb01-8f53-47a9-9f13-2a15b49031ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:34:25.038197 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:34:25.272728 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 711/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 06:34:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:34:25.425916 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf41ecb-39a2-4b98-8623-d40134b59da6 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:34:25.427492 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-adf41ecb-39a2-4b98-8623-d40134b59da6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:34:25.588667 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf41ecb-39a2-4b98-8623-d40134b59da6 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:34:25.612512 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:34:25.707962 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0d87d3-c55c-4533-bb40-52c42b0da604 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4f49c1b08c460f8ba6c451c9c92c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:34:25.708579 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0d87d3-c55c-4533-bb40-52c42b0da604 service nova] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:25.708864 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0d87d3-c55c-4533-bb40-52c42b0da604 service nova] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:25.709243 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0d87d3-c55c-4533-bb40-52c42b0da604 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:25.710855 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-9f0d87d3-c55c-4533-bb40-52c42b0da604 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:34:25.811399 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0d87d3-c55c-4533-bb40-52c42b0da604 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:34:25.813027 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0d87d3-c55c-4533-bb40-52c42b0da604 service nova] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:34:25.832571 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 729/1440] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:34:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:34:27.662064 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccbd43c-8360-4c90-8fe9-47f9d47b3a28 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:27.662576 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccbd43c-8360-4c90-8fe9-47f9d47b3a28 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:27.662576 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccbd43c-8360-4c90-8fe9-47f9d47b3a28 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:27.663922 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 712/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 06:34:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:34:27.669501 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38436aee-2e27-437e-a579-82d49e68e99e None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:27.669908 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38436aee-2e27-437e-a579-82d49e68e99e None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:27.670063 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38436aee-2e27-437e-a579-82d49e68e99e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:27.712882 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:34:27.739812 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-38436aee-2e27-437e-a579-82d49e68e99e None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:34:27.741111 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-38436aee-2e27-437e-a579-82d49e68e99e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:34:27.841761 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 730/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 06:34:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:34:28.123192 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccbd43c-8360-4c90-8fe9-47f9d47b3a28 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:34:28.124719 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-8ccbd43c-8360-4c90-8fe9-47f9d47b3a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:34:28.231037 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccbd43c-8360-4c90-8fe9-47f9d47b3a28 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:34:28.264831 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:34:28.307968 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df866f2-d1bb-4164-9add-80619f1a6273 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:34:28.308584 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df866f2-d1bb-4164-9add-80619f1a6273 service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:28.308862 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df866f2-d1bb-4164-9add-80619f1a6273 service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:28.309137 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df866f2-d1bb-4164-9add-80619f1a6273 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:28.310825 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-4df866f2-d1bb-4164-9add-80619f1a6273 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:34:28.424742 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df866f2-d1bb-4164-9add-80619f1a6273 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:34:28.426135 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df866f2-d1bb-4164-9add-80619f1a6273 service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:34:28.444738 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 713/1443] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:34:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:34:30.498193 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f54e9e5-c6c2-4f0c-b112-d1d0159764cc None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:30.498766 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f54e9e5-c6c2-4f0c-b112-d1d0159764cc None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:30.498766 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f54e9e5-c6c2-4f0c-b112-d1d0159764cc None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:30.500113 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 731/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 06:34:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:34:30.505640 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a259626-aa2f-4fca-9e70-9489fef80e9e None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:30.506060 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a259626-aa2f-4fca-9e70-9489fef80e9e None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:30.506159 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a259626-aa2f-4fca-9e70-9489fef80e9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:30.571916 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-3a259626-aa2f-4fca-9e70-9489fef80e9e None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:34:30.573136 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-3a259626-aa2f-4fca-9e70-9489fef80e9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:34:30.586337 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:34:30.672959 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 714/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 06:34:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:34:30.958275 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f54e9e5-c6c2-4f0c-b112-d1d0159764cc None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:34:30.959832 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-3f54e9e5-c6c2-4f0c-b112-d1d0159764cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:34:31.190501 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f54e9e5-c6c2-4f0c-b112-d1d0159764cc None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:34:31.202768 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:34:31.265977 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13868c6-4e97-4569-9039-683e69d37fc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:34:31.266585 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13868c6-4e97-4569-9039-683e69d37fc8 service neutron] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:31.266866 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13868c6-4e97-4569-9039-683e69d37fc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:31.267137 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13868c6-4e97-4569-9039-683e69d37fc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:31.269866 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-c13868c6-4e97-4569-9039-683e69d37fc8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:34:31.379044 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13868c6-4e97-4569-9039-683e69d37fc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:34:31.379821 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13868c6-4e97-4569-9039-683e69d37fc8 service neutron] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:34:31.398683 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 732/1446] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:34:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:34:33.867021 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8591e13-67ed-4058-8fbc-6b5d739cc2f3 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:33.867575 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8591e13-67ed-4058-8fbc-6b5d739cc2f3 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:33.867575 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8591e13-67ed-4058-8fbc-6b5d739cc2f3 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:33.868914 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 715/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 06:34:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:34:33.874489 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330b4786-f599-4cc0-968b-f03ccc27e55a None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:33.874890 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330b4786-f599-4cc0-968b-f03ccc27e55a None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:33.875026 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330b4786-f599-4cc0-968b-f03ccc27e55a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:33.923818 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:34:33.942980 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-330b4786-f599-4cc0-968b-f03ccc27e55a None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:34:33.944198 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-330b4786-f599-4cc0-968b-f03ccc27e55a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:34:34.039287 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 733/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 06:34:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:34:34.322376 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8591e13-67ed-4058-8fbc-6b5d739cc2f3 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:34:34.323913 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-b8591e13-67ed-4058-8fbc-6b5d739cc2f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:34:34.428024 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8591e13-67ed-4058-8fbc-6b5d739cc2f3 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:34:34.467655 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:34:34.499841 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a363b1-8a00-4320-9e58-ae0a2bdb13ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:34:34.500433 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a363b1-8a00-4320-9e58-ae0a2bdb13ce service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:34.500719 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a363b1-8a00-4320-9e58-ae0a2bdb13ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:34.500976 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a363b1-8a00-4320-9e58-ae0a2bdb13ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:34.503555 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-a1a363b1-8a00-4320-9e58-ae0a2bdb13ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:34:34.611995 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a363b1-8a00-4320-9e58-ae0a2bdb13ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:34:34.613526 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a363b1-8a00-4320-9e58-ae0a2bdb13ce service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:34:34.632944 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 716/1449] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:34:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:34:36.751033 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed990bfe-3f8b-4477-86fd-9d3e88521df8 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:36.751570 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed990bfe-3f8b-4477-86fd-9d3e88521df8 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:36.751570 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed990bfe-3f8b-4477-86fd-9d3e88521df8 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:36.753418 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 734/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 06:34:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:34:36.758927 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8b0fef-2f71-422b-b21d-bbad5d301f77 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:36.759365 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8b0fef-2f71-422b-b21d-bbad5d301f77 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:36.759477 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8b0fef-2f71-422b-b21d-bbad5d301f77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:36.825441 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-da8b0fef-2f71-422b-b21d-bbad5d301f77 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:34:36.826673 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-da8b0fef-2f71-422b-b21d-bbad5d301f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:34:36.850474 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:34:36.931308 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 717/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 06:34:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:34:37.221372 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed990bfe-3f8b-4477-86fd-9d3e88521df8 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:34:37.228639 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-ed990bfe-3f8b-4477-86fd-9d3e88521df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:34:37.353110 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed990bfe-3f8b-4477-86fd-9d3e88521df8 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:34:37.379591 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:34:37.450526 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e634a2e-d70e-40c0-89ea-4071913a9689 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:34:37.451125 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e634a2e-d70e-40c0-89ea-4071913a9689 service neutron] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:37.451399 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e634a2e-d70e-40c0-89ea-4071913a9689 service neutron] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:37.451667 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e634a2e-d70e-40c0-89ea-4071913a9689 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:37.453301 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-9e634a2e-d70e-40c0-89ea-4071913a9689 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:34:37.573448 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e634a2e-d70e-40c0-89ea-4071913a9689 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:34:37.574800 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e634a2e-d70e-40c0-89ea-4071913a9689 service neutron] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:34:37.592254 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 735/1452] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:34:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:34:39.580823 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f24bcb9-3f1c-4946-9033-f7889ddb7104 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:39.581420 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f24bcb9-3f1c-4946-9033-f7889ddb7104 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:39.581420 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f24bcb9-3f1c-4946-9033-f7889ddb7104 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:39.582770 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 718/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 06:34:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:34:39.588244 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d76fb59-ec38-4f2f-85a9-748fc3f2bf81 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:39.588650 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d76fb59-ec38-4f2f-85a9-748fc3f2bf81 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:39.588799 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d76fb59-ec38-4f2f-85a9-748fc3f2bf81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:39.655164 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:34:39.657819 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-3d76fb59-ec38-4f2f-85a9-748fc3f2bf81 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:34:39.659140 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-3d76fb59-ec38-4f2f-85a9-748fc3f2bf81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:34:39.756248 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 736/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 06:34:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:34:40.043961 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f24bcb9-3f1c-4946-9033-f7889ddb7104 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:34:40.045556 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-1f24bcb9-3f1c-4946-9033-f7889ddb7104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:34:40.148207 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f24bcb9-3f1c-4946-9033-f7889ddb7104 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:34:40.196673 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:34:40.221751 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fd1d7d-4d2b-4438-9c3f-906ad9a6d692 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:34:40.223974 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fd1d7d-4d2b-4438-9c3f-906ad9a6d692 service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:40.223974 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fd1d7d-4d2b-4438-9c3f-906ad9a6d692 service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:40.223974 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fd1d7d-4d2b-4438-9c3f-906ad9a6d692 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:40.224556 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-e1fd1d7d-4d2b-4438-9c3f-906ad9a6d692 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:34:40.334283 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fd1d7d-4d2b-4438-9c3f-906ad9a6d692 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:34:40.335707 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fd1d7d-4d2b-4438-9c3f-906ad9a6d692 service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:34:40.354622 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 719/1455] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:34:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:34:42.351740 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0552ce1f-9310-45ff-ac0e-66685dd0f3bc None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:42.352292 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0552ce1f-9310-45ff-ac0e-66685dd0f3bc None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:42.352292 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0552ce1f-9310-45ff-ac0e-66685dd0f3bc None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:42.353663 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 737/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 06:34:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:34:42.359068 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f853d1-5617-40bc-a803-e464fa448f1b None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:42.359471 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f853d1-5617-40bc-a803-e464fa448f1b None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:42.359614 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f853d1-5617-40bc-a803-e464fa448f1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:42.424923 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-f2f853d1-5617-40bc-a803-e464fa448f1b None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:34:42.426167 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-f2f853d1-5617-40bc-a803-e464fa448f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:34:42.457080 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:34:42.526504 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 720/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 06:34:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:34:42.810724 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0552ce1f-9310-45ff-ac0e-66685dd0f3bc None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:34:42.812277 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-0552ce1f-9310-45ff-ac0e-66685dd0f3bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:34:42.919659 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0552ce1f-9310-45ff-ac0e-66685dd0f3bc None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:34:42.949191 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:34:42.993818 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5f71ac-52ee-487f-adf8-e91e2b37feb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:34:42.994424 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5f71ac-52ee-487f-adf8-e91e2b37feb3 service neutron] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:42.994702 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5f71ac-52ee-487f-adf8-e91e2b37feb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:42.994970 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5f71ac-52ee-487f-adf8-e91e2b37feb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:42.996564 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-7c5f71ac-52ee-487f-adf8-e91e2b37feb3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:34:43.104720 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5f71ac-52ee-487f-adf8-e91e2b37feb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:34:43.106186 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5f71ac-52ee-487f-adf8-e91e2b37feb3 service neutron] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:34:43.124130 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 738/1458] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:34:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:34:45.134810 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b0940d-fa1b-4cc4-ba77-ce78fee877b3 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:45.135356 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b0940d-fa1b-4cc4-ba77-ce78fee877b3 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:45.135356 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b0940d-fa1b-4cc4-ba77-ce78fee877b3 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:45.136706 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 721/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 06:34:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:34:45.142141 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cdfd5d-bb9d-4896-824d-b25a90696351 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:45.143028 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cdfd5d-bb9d-4896-824d-b25a90696351 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:45.143198 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cdfd5d-bb9d-4896-824d-b25a90696351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:45.209141 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-a6cdfd5d-bb9d-4896-824d-b25a90696351 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:34:45.210374 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-a6cdfd5d-bb9d-4896-824d-b25a90696351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:34:45.218659 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:34:45.311278 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 739/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 06:34:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:34:45.601866 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b0940d-fa1b-4cc4-ba77-ce78fee877b3 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:34:45.603569 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-b7b0940d-fa1b-4cc4-ba77-ce78fee877b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:34:45.715237 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b0940d-fa1b-4cc4-ba77-ce78fee877b3 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:34:45.777320 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:34:45.797712 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bfb688-5a3d-4b9d-8728-8f1db10acf53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:34:45.798328 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bfb688-5a3d-4b9d-8728-8f1db10acf53 service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:45.798609 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bfb688-5a3d-4b9d-8728-8f1db10acf53 service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:45.799445 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bfb688-5a3d-4b9d-8728-8f1db10acf53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:45.801110 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-69bfb688-5a3d-4b9d-8728-8f1db10acf53 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:34:45.922064 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bfb688-5a3d-4b9d-8728-8f1db10acf53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:34:45.923520 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bfb688-5a3d-4b9d-8728-8f1db10acf53 service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:34:45.944388 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 722/1461] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:34:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:34:47.949023 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af319c-a67a-4154-aaa6-8c290bae5367 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:47.949667 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af319c-a67a-4154-aaa6-8c290bae5367 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:47.949667 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af319c-a67a-4154-aaa6-8c290bae5367 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:47.950948 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 740/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 06:34:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:34:47.957102 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d57629-651d-4b2e-bd21-397c3b6fd983 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:47.957615 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d57629-651d-4b2e-bd21-397c3b6fd983 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:47.957732 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d57629-651d-4b2e-bd21-397c3b6fd983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:48.034449 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-99d57629-651d-4b2e-bd21-397c3b6fd983 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:34:48.035667 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-99d57629-651d-4b2e-bd21-397c3b6fd983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:34:48.076335 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:34:48.140761 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 723/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 06:34:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:34:48.430462 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34af319c-a67a-4154-aaa6-8c290bae5367 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:34:48.432017 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-34af319c-a67a-4154-aaa6-8c290bae5367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:34:48.541285 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34af319c-a67a-4154-aaa6-8c290bae5367 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:34:48.581907 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:34:48.619501 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c225c8ca-4f63-4c6d-8c40-e92afdbca46e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:34:48.620113 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c225c8ca-4f63-4c6d-8c40-e92afdbca46e service neutron] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:48.620394 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c225c8ca-4f63-4c6d-8c40-e92afdbca46e service neutron] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:48.620676 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c225c8ca-4f63-4c6d-8c40-e92afdbca46e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:48.628260 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-c225c8ca-4f63-4c6d-8c40-e92afdbca46e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:34:48.740055 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c225c8ca-4f63-4c6d-8c40-e92afdbca46e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:34:48.741458 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c225c8ca-4f63-4c6d-8c40-e92afdbca46e service neutron] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:34:48.761242 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 741/1464] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:34:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:34:50.788575 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4653d7-e302-4bd9-87b4-afabfa3f6792 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:50.789139 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4653d7-e302-4bd9-87b4-afabfa3f6792 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:50.789139 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4653d7-e302-4bd9-87b4-afabfa3f6792 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:50.790504 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 724/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 06:34:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:34:50.796063 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6d5aba-60fc-496c-96f5-5be04bbc48db None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:50.796478 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6d5aba-60fc-496c-96f5-5be04bbc48db None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:50.796608 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6d5aba-60fc-496c-96f5-5be04bbc48db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:50.866202 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-4c6d5aba-60fc-496c-96f5-5be04bbc48db None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:34:50.867460 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-4c6d5aba-60fc-496c-96f5-5be04bbc48db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:34:50.887000 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:34:50.969758 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 742/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 06:34:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:34:51.310205 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4653d7-e302-4bd9-87b4-afabfa3f6792 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:34:51.311761 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-2e4653d7-e302-4bd9-87b4-afabfa3f6792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:34:51.424498 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4653d7-e302-4bd9-87b4-afabfa3f6792 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:34:51.490584 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:34:51.497468 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7600867-21e9-410a-833c-ddfcb0d03f69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:34:51.498103 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7600867-21e9-410a-833c-ddfcb0d03f69 service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:51.498379 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7600867-21e9-410a-833c-ddfcb0d03f69 service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:51.498646 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7600867-21e9-410a-833c-ddfcb0d03f69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:51.500285 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-f7600867-21e9-410a-833c-ddfcb0d03f69 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:34:51.613449 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7600867-21e9-410a-833c-ddfcb0d03f69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:34:51.614880 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7600867-21e9-410a-833c-ddfcb0d03f69 service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:34:51.634312 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 725/1467] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:34:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:34:53.597472 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146daa41-32d9-47fe-9741-56cde4cae37d None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:53.598045 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146daa41-32d9-47fe-9741-56cde4cae37d None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:53.598290 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146daa41-32d9-47fe-9741-56cde4cae37d None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:53.599654 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 743/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 06:34:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:34:53.605012 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a8bba4-1019-44aa-b97e-803e064bf7d9 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:53.605451 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a8bba4-1019-44aa-b97e-803e064bf7d9 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:53.605624 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a8bba4-1019-44aa-b97e-803e064bf7d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:53.672674 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-01a8bba4-1019-44aa-b97e-803e064bf7d9 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:34:53.674288 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-01a8bba4-1019-44aa-b97e-803e064bf7d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:34:53.729804 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:34:53.780292 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 726/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 06:34:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:34:54.063420 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146daa41-32d9-47fe-9741-56cde4cae37d None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:34:54.065677 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-146daa41-32d9-47fe-9741-56cde4cae37d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:34:54.169938 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146daa41-32d9-47fe-9741-56cde4cae37d None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:34:54.218896 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:34:54.245934 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21df6c62-ccf8-4467-afda-1c652f9e2700 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:34:54.246547 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21df6c62-ccf8-4467-afda-1c652f9e2700 service neutron] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:34:54.246855 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21df6c62-ccf8-4467-afda-1c652f9e2700 service neutron] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:34:54.247129 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21df6c62-ccf8-4467-afda-1c652f9e2700 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:34:54.248782 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-21df6c62-ccf8-4467-afda-1c652f9e2700 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:34:54.365422 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21df6c62-ccf8-4467-afda-1c652f9e2700 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:34:54.366866 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21df6c62-ccf8-4467-afda-1c652f9e2700 service neutron] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:34:54.387104 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 744/1470] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:34:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:35:03.778566 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=22 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:35:04.384408 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=66 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:35:14.264754 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58ed96d-d3c2-40ca-a3d3-d6eee8a0167e None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:35:14.265418 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58ed96d-d3c2-40ca-a3d3-d6eee8a0167e None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:35:14.265418 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58ed96d-d3c2-40ca-a3d3-d6eee8a0167e None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:35:14.266899 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 727/1471] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:35:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:35:14.272236 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97f0671-9559-42e7-af0f-d5899b267b07 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:35:14.272637 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97f0671-9559-42e7-af0f-d5899b267b07 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:35:14.272785 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97f0671-9559-42e7-af0f-d5899b267b07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:35:14.342316 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-e97f0671-9559-42e7-af0f-d5899b267b07 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:35:14.343512 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-e97f0671-9559-42e7-af0f-d5899b267b07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:35:14.438898 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 745/1472] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:35:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:35:14.736651 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07705523-587f-498c-9dbf-78262f16db2f None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:35:14.736933 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07705523-587f-498c-9dbf-78262f16db2f None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:35:14.737241 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07705523-587f-498c-9dbf-78262f16db2f None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:35:14.738566 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 728/1473] 10.222.0.47 () {54 vars in 967 bytes} [Wed Aug 24 06:35:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:35:14.743586 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92421cd4-53f6-464d-b8f9-598849746a1b None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:35:14.743897 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92421cd4-53f6-464d-b8f9-598849746a1b None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:35:14.744180 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92421cd4-53f6-464d-b8f9-598849746a1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:35:14.831019 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-92421cd4-53f6-464d-b8f9-598849746a1b None None] MFA Rules not processed for user `31b331b6ddfe46aeae7b52fcbbc27771`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:35:14.832236 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-92421cd4-53f6-464d-b8f9-598849746a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:35:14.855705 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:35:14.917224 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 746/1474] 10.222.0.47 () {58 vars in 1064 bytes} [Wed Aug 24 06:35:14 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:35:14.924990 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07705523-587f-498c-9dbf-78262f16db2f None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:35:14.926700 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-07705523-587f-498c-9dbf-78262f16db2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:35:15.039435 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07705523-587f-498c-9dbf-78262f16db2f None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:35:15.117557 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644e3ffc-19a4-42b0-99c6-3ebf8d8ae8b6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b331b6ddfe46aeae7b52fcbbc27771', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:35:15.118172 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644e3ffc-19a4-42b0-99c6-3ebf8d8ae8b6 service glance] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:35:15.118452 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644e3ffc-19a4-42b0-99c6-3ebf8d8ae8b6 service glance] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:35:15.118724 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644e3ffc-19a4-42b0-99c6-3ebf8d8ae8b6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:35:15.120339 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-644e3ffc-19a4-42b0-99c6-3ebf8d8ae8b6 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:35:15.177827 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:35:15.236568 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644e3ffc-19a4-42b0-99c6-3ebf8d8ae8b6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:35:15.238029 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644e3ffc-19a4-42b0-99c6-3ebf8d8ae8b6 service glance] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:35:15.258607 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 729/1475] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:35:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:35:17.274365 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066a6765-41d6-4254-b2bc-2e61dab9d9f2 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:35:17.274934 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066a6765-41d6-4254-b2bc-2e61dab9d9f2 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:35:17.274934 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066a6765-41d6-4254-b2bc-2e61dab9d9f2 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:35:17.276268 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 747/1476] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:35:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:35:17.282322 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c7dd79-65b6-47a5-8d80-e1beae14ab2f None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:35:17.282994 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c7dd79-65b6-47a5-8d80-e1beae14ab2f None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:35:17.282994 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c7dd79-65b6-47a5-8d80-e1beae14ab2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:35:17.352669 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-59c7dd79-65b6-47a5-8d80-e1beae14ab2f None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:35:17.353948 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-59c7dd79-65b6-47a5-8d80-e1beae14ab2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:35:17.460479 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 730/1477] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:35:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:35:17.733390 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066a6765-41d6-4254-b2bc-2e61dab9d9f2 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:35:17.735318 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-066a6765-41d6-4254-b2bc-2e61dab9d9f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:35:17.874201 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066a6765-41d6-4254-b2bc-2e61dab9d9f2 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:35:17.924887 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:35:17.970965 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee13286c-7d82-4021-af36-117879b202b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:35:17.971617 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee13286c-7d82-4021-af36-117879b202b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:35:17.971891 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee13286c-7d82-4021-af36-117879b202b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:35:17.972158 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee13286c-7d82-4021-af36-117879b202b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:35:17.972797 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee13286c-7d82-4021-af36-117879b202b4 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:35:17.974557 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: WARNING py.warnings [None req-ee13286c-7d82-4021-af36-117879b202b4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 06:35:17.974557 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: warnings.warn(msg) Aug 24 06:35:17.974557 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: Aug 24 06:35:17.976447 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee13286c-7d82-4021-af36-117879b202b4 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:35:17.994914 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 748/1478] 10.222.0.47 () {56 vars in 1108 bytes} [Wed Aug 24 06:35:17 2022] GET /identity/v3/projects => generated 2553 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:35:19.952527 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942703c7-3c86-4058-9669-dd2b79dd3a82 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:35:19.953118 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942703c7-3c86-4058-9669-dd2b79dd3a82 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:35:19.953249 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942703c7-3c86-4058-9669-dd2b79dd3a82 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:35:19.954632 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 731/1479] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:35:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:35:19.959883 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0e5aea-79cc-45eb-90cf-960664e4289b None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:35:19.960339 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0e5aea-79cc-45eb-90cf-960664e4289b None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:35:19.960466 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0e5aea-79cc-45eb-90cf-960664e4289b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:35:20.030619 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-7c0e5aea-79cc-45eb-90cf-960664e4289b None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:35:20.032460 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-7c0e5aea-79cc-45eb-90cf-960664e4289b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:35:20.129027 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 749/1480] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:35:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:35:20.429029 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942703c7-3c86-4058-9669-dd2b79dd3a82 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:35:20.430692 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-942703c7-3c86-4058-9669-dd2b79dd3a82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:35:20.441547 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:35:20.587738 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942703c7-3c86-4058-9669-dd2b79dd3a82 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:35:20.683670 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:35:20.720872 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a51b4f9-7911-4629-b1f5-a0f938a3290d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4f49c1b08c460f8ba6c451c9c92c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:35:20.721523 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a51b4f9-7911-4629-b1f5-a0f938a3290d service nova] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:35:20.721809 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a51b4f9-7911-4629-b1f5-a0f938a3290d service nova] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:35:20.722079 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a51b4f9-7911-4629-b1f5-a0f938a3290d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:35:20.723730 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-7a51b4f9-7911-4629-b1f5-a0f938a3290d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:35:20.828650 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a51b4f9-7911-4629-b1f5-a0f938a3290d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:35:20.830281 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a51b4f9-7911-4629-b1f5-a0f938a3290d service nova] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:35:20.849526 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 732/1481] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:35:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:35:23.000292 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc508876-4ee4-4835-9d8c-959d8c56f286 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:35:23.000868 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc508876-4ee4-4835-9d8c-959d8c56f286 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:35:23.000981 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc508876-4ee4-4835-9d8c-959d8c56f286 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:35:23.002313 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 750/1482] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:35:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:35:23.007690 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bfd6de-321f-4875-bd65-3af94fa45f2a None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:35:23.008116 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bfd6de-321f-4875-bd65-3af94fa45f2a None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:35:23.008245 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bfd6de-321f-4875-bd65-3af94fa45f2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:35:23.074474 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-b3bfd6de-321f-4875-bd65-3af94fa45f2a None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:35:23.075685 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-b3bfd6de-321f-4875-bd65-3af94fa45f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:35:23.123585 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:35:23.174295 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 733/1483] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:35:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:35:23.468200 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc508876-4ee4-4835-9d8c-959d8c56f286 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:35:23.470868 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-bc508876-4ee4-4835-9d8c-959d8c56f286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:35:23.496756 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:35:23.594930 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc508876-4ee4-4835-9d8c-959d8c56f286 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:35:23.673706 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d6f918-fff8-4758-aa0e-4c82cdb85d99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:35:23.674203 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d6f918-fff8-4758-aa0e-4c82cdb85d99 service neutron] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:35:23.674478 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d6f918-fff8-4758-aa0e-4c82cdb85d99 service neutron] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:35:23.674744 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d6f918-fff8-4758-aa0e-4c82cdb85d99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:35:23.676367 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-a5d6f918-fff8-4758-aa0e-4c82cdb85d99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:35:23.754407 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:35:23.790320 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d6f918-fff8-4758-aa0e-4c82cdb85d99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:35:23.792498 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d6f918-fff8-4758-aa0e-4c82cdb85d99 service neutron] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:35:23.811672 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 751/1484] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:35:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:35:25.804540 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281ced14-fd46-4de6-b5a8-e24b11074aec None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:35:25.805118 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281ced14-fd46-4de6-b5a8-e24b11074aec None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:35:25.805118 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281ced14-fd46-4de6-b5a8-e24b11074aec None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:35:25.806454 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 734/1485] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:35:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:35:25.811695 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b352bc-0fac-41fe-8516-9546e959fb8c None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:35:25.812790 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b352bc-0fac-41fe-8516-9546e959fb8c None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:35:25.812920 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b352bc-0fac-41fe-8516-9546e959fb8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:35:25.879144 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-b9b352bc-0fac-41fe-8516-9546e959fb8c None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:35:25.880376 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-b9b352bc-0fac-41fe-8516-9546e959fb8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:35:25.978659 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 752/1486] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:35:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:35:26.270853 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281ced14-fd46-4de6-b5a8-e24b11074aec None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:35:26.272388 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-281ced14-fd46-4de6-b5a8-e24b11074aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:35:26.381691 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281ced14-fd46-4de6-b5a8-e24b11074aec None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:35:26.460032 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:35:26.461822 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437cf361-030d-4a8e-ab1f-1d9a8298125d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:35:26.462444 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437cf361-030d-4a8e-ab1f-1d9a8298125d service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:35:26.462718 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437cf361-030d-4a8e-ab1f-1d9a8298125d service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:35:26.462982 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437cf361-030d-4a8e-ab1f-1d9a8298125d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:35:26.464603 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-437cf361-030d-4a8e-ab1f-1d9a8298125d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:35:26.611443 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437cf361-030d-4a8e-ab1f-1d9a8298125d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:35:26.612896 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437cf361-030d-4a8e-ab1f-1d9a8298125d service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:35:26.641158 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 735/1487] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:35:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:35:28.763781 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242afc26-8dce-42ce-9056-b4148bb34766 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:35:28.764340 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242afc26-8dce-42ce-9056-b4148bb34766 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:35:28.764340 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242afc26-8dce-42ce-9056-b4148bb34766 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:35:28.766281 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 753/1488] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:35:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:35:28.772188 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a26268-bff5-476e-b793-9744174accb1 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:35:28.772682 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a26268-bff5-476e-b793-9744174accb1 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:35:28.772786 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a26268-bff5-476e-b793-9744174accb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:35:28.845365 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-66a26268-bff5-476e-b793-9744174accb1 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:35:28.846575 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-66a26268-bff5-476e-b793-9744174accb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:35:28.902245 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:35:28.949551 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 736/1489] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:35:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:35:29.239932 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242afc26-8dce-42ce-9056-b4148bb34766 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:35:29.241498 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-242afc26-8dce-42ce-9056-b4148bb34766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:35:29.278535 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:35:29.351693 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242afc26-8dce-42ce-9056-b4148bb34766 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:35:29.428212 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8c476e-293a-402d-bd7c-5097ea23a752 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:35:29.428816 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8c476e-293a-402d-bd7c-5097ea23a752 service neutron] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:35:29.429092 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8c476e-293a-402d-bd7c-5097ea23a752 service neutron] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:35:29.429389 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8c476e-293a-402d-bd7c-5097ea23a752 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:35:29.430982 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-9d8c476e-293a-402d-bd7c-5097ea23a752 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:35:29.512534 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:35:29.544236 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8c476e-293a-402d-bd7c-5097ea23a752 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:35:29.545803 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8c476e-293a-402d-bd7c-5097ea23a752 service neutron] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:35:29.564319 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 754/1490] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:35:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:35:32.106849 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f670c62-92c4-4e60-8643-fa8337cf40f8 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:35:32.107432 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f670c62-92c4-4e60-8643-fa8337cf40f8 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:35:32.107432 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f670c62-92c4-4e60-8643-fa8337cf40f8 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:35:32.109770 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 737/1491] 10.222.0.47 () {54 vars in 927 bytes} [Wed Aug 24 06:35:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:35:32.115414 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ad21ef-166d-4061-81f0-94c8f24ef42d None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:35:32.115868 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ad21ef-166d-4061-81f0-94c8f24ef42d None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:35:32.115971 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ad21ef-166d-4061-81f0-94c8f24ef42d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:35:32.181640 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-b6ad21ef-166d-4061-81f0-94c8f24ef42d None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:35:32.182842 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-b6ad21ef-166d-4061-81f0-94c8f24ef42d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:35:32.278608 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 755/1492] 10.222.0.47 () {58 vars in 1024 bytes} [Wed Aug 24 06:35:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:35:32.580060 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f670c62-92c4-4e60-8643-fa8337cf40f8 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:35:32.581777 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-2f670c62-92c4-4e60-8643-fa8337cf40f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:35:32.689072 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f670c62-92c4-4e60-8643-fa8337cf40f8 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:35:32.761718 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df130526-b3f2-45cd-978d-7c0f21295ad6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:35:32.762347 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df130526-b3f2-45cd-978d-7c0f21295ad6 service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:35:32.762630 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df130526-b3f2-45cd-978d-7c0f21295ad6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:35:32.762894 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df130526-b3f2-45cd-978d-7c0f21295ad6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:35:32.764530 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-df130526-b3f2-45cd-978d-7c0f21295ad6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:35:32.774412 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:35:32.874813 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df130526-b3f2-45cd-978d-7c0f21295ad6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:35:32.876333 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df130526-b3f2-45cd-978d-7c0f21295ad6 service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:35:32.894997 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 738/1493] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:35:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:35:42.276789 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=82 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:35:42.892231 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=50 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:36:55.166239 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebca2d2b-e129-48ad-a526-e9ea535337a0 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:55.166862 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebca2d2b-e129-48ad-a526-e9ea535337a0 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:55.166981 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebca2d2b-e129-48ad-a526-e9ea535337a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:55.251668 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-ebca2d2b-e129-48ad-a526-e9ea535337a0 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:36:55.253090 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-ebca2d2b-e129-48ad-a526-e9ea535337a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:36:55.352858 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 756/1494] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:36:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:36:55.361624 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44235d50-b462-438f-ba6d-297e6ca956f7 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:55.362146 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44235d50-b462-438f-ba6d-297e6ca956f7 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:55.362248 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44235d50-b462-438f-ba6d-297e6ca956f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:55.449495 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-44235d50-b462-438f-ba6d-297e6ca956f7 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:36:55.450672 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-44235d50-b462-438f-ba6d-297e6ca956f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:36:55.549475 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 739/1495] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:36:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:36:55.554189 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebca2d2b-e129-48ad-a526-e9ea535337a0 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:36:55.555413 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-ebca2d2b-e129-48ad-a526-e9ea535337a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:36:55.567121 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebca2d2b-e129-48ad-a526-e9ea535337a0 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:36:55.632908 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:36:55.669555 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b00c037-85e6-4e26-bfa5-7976e8dcd36d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:36:55.670717 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b00c037-85e6-4e26-bfa5-7976e8dcd36d admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:55.670994 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b00c037-85e6-4e26-bfa5-7976e8dcd36d admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:55.671284 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b00c037-85e6-4e26-bfa5-7976e8dcd36d admin admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:55.672030 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b00c037-85e6-4e26-bfa5-7976e8dcd36d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:36:55.672326 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b00c037-85e6-4e26-bfa5-7976e8dcd36d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:36:55.673357 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: WARNING py.warnings [None req-4b00c037-85e6-4e26-bfa5-7976e8dcd36d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 06:36:55.673357 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: warnings.warn(msg) Aug 24 06:36:55.673357 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: Aug 24 06:36:55.674238 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b00c037-85e6-4e26-bfa5-7976e8dcd36d admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:36:55.687131 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 757/1496] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 24 06:36:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:36:55.695729 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44235d50-b462-438f-ba6d-297e6ca956f7 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:36:55.705622 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44235d50-b462-438f-ba6d-297e6ca956f7 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:36:55.768895 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:36:55.805331 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db0157c-29cd-4cd4-8d2e-5e1553445bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:36:55.805941 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db0157c-29cd-4cd4-8d2e-5e1553445bb9 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:55.806261 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db0157c-29cd-4cd4-8d2e-5e1553445bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:55.806536 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db0157c-29cd-4cd4-8d2e-5e1553445bb9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:55.807158 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db0157c-29cd-4cd4-8d2e-5e1553445bb9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:36:55.808944 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: WARNING py.warnings [None req-5db0157c-29cd-4cd4-8d2e-5e1553445bb9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 06:36:55.808944 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: warnings.warn(msg) Aug 24 06:36:55.808944 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: Aug 24 06:36:55.810344 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db0157c-29cd-4cd4-8d2e-5e1553445bb9 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:36:55.963969 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 740/1497] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:36:55 2022] POST /identity/v3/projects => generated 379 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:36:55.969315 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b00c037-85e6-4e26-bfa5-7976e8dcd36d admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:36:55.970925 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-4b00c037-85e6-4e26-bfa5-7976e8dcd36d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:36:56.024473 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b00c037-85e6-4e26-bfa5-7976e8dcd36d admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:36:56.151952 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5250f72e-86ad-4b11-9cee-b25ea57eea06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:36:56.152599 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5250f72e-86ad-4b11-9cee-b25ea57eea06 admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:56.152898 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5250f72e-86ad-4b11-9cee-b25ea57eea06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:56.153213 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5250f72e-86ad-4b11-9cee-b25ea57eea06 admin admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:56.154702 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5250f72e-86ad-4b11-9cee-b25ea57eea06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:36:56.156073 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: WARNING py.warnings [None req-5250f72e-86ad-4b11-9cee-b25ea57eea06 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 06:36:56.156073 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: warnings.warn(msg) Aug 24 06:36:56.156073 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: Aug 24 06:36:56.157322 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5250f72e-86ad-4b11-9cee-b25ea57eea06 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:36:56.267572 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 758/1498] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:36:55 2022] POST /identity/v3/users => generated 290 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:36:56.271821 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db0157c-29cd-4cd4-8d2e-5e1553445bb9 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:36:56.273409 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-5db0157c-29cd-4cd4-8d2e-5e1553445bb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:36:56.338844 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db0157c-29cd-4cd4-8d2e-5e1553445bb9 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:36:56.425556 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:36:56.436910 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e133a94-3f97-4f19-89f5-ac9fd5f38587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:36:56.439573 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e133a94-3f97-4f19-89f5-ac9fd5f38587 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:56.439573 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e133a94-3f97-4f19-89f5-ac9fd5f38587 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:56.439573 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e133a94-3f97-4f19-89f5-ac9fd5f38587 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:56.439573 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e133a94-3f97-4f19-89f5-ac9fd5f38587 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:36:56.440054 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: WARNING py.warnings [None req-3e133a94-3f97-4f19-89f5-ac9fd5f38587 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 06:36:56.440054 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: warnings.warn(msg) Aug 24 06:36:56.440054 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: Aug 24 06:36:56.440570 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e133a94-3f97-4f19-89f5-ac9fd5f38587 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:36:56.454641 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 741/1499] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:36:56 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:36:56.459236 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5250f72e-86ad-4b11-9cee-b25ea57eea06 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:36:56.460818 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-5250f72e-86ad-4b11-9cee-b25ea57eea06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:36:56.473159 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5250f72e-86ad-4b11-9cee-b25ea57eea06 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:36:56.501614 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:36:56.581965 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64d7819-8b1c-4ce5-83c0-e71783508ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:36:56.582573 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64d7819-8b1c-4ce5-83c0-e71783508ea3 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:56.582901 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64d7819-8b1c-4ce5-83c0-e71783508ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:56.583217 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64d7819-8b1c-4ce5-83c0-e71783508ea3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:56.584009 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64d7819-8b1c-4ce5-83c0-e71783508ea3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:36:56.585095 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: WARNING py.warnings [None req-c64d7819-8b1c-4ce5-83c0-e71783508ea3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 06:36:56.585095 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: warnings.warn(msg) Aug 24 06:36:56.585095 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: Aug 24 06:36:56.586029 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64d7819-8b1c-4ce5-83c0-e71783508ea3 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:36:56.601552 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 759/1500] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:36:56 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:36:56.606087 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e133a94-3f97-4f19-89f5-ac9fd5f38587 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:36:56.608749 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-3e133a94-3f97-4f19-89f5-ac9fd5f38587 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:36:56.650245 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e133a94-3f97-4f19-89f5-ac9fd5f38587 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:36:56.759502 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594c38a7-d3d9-42b7-927e-b95f1f8ba4f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:36:56.760176 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594c38a7-d3d9-42b7-927e-b95f1f8ba4f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:56.760456 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594c38a7-d3d9-42b7-927e-b95f1f8ba4f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:56.760731 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594c38a7-d3d9-42b7-927e-b95f1f8ba4f5 admin admin] PATH_INFO: `/v3/projects/ab456fa0620847b49b1433670a448d1f/users/3cd734291818490db32874612e1529ed/roles/e1f9b167c51c4a388cbc9594db8a465d` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:56.786066 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594c38a7-d3d9-42b7-927e-b95f1f8ba4f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab456fa0620847b49b1433670a448d1f, user_id=3cd734291818490db32874612e1529ed, role_id=e1f9b167c51c4a388cbc9594db8a465d)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:36:56.787102 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: WARNING py.warnings [None req-594c38a7-d3d9-42b7-927e-b95f1f8ba4f5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 06:36:56.787102 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: warnings.warn(msg) Aug 24 06:36:56.787102 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: Aug 24 06:36:56.789262 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594c38a7-d3d9-42b7-927e-b95f1f8ba4f5 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:36:56.896167 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 742/1501] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:36:56 2022] PUT /identity/v3/projects/ab456fa0620847b49b1433670a448d1f/users/3cd734291818490db32874612e1529ed/roles/e1f9b167c51c4a388cbc9594db8a465d => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:36:56.909091 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e5b22d-74ef-4c25-9bb6-8467db05e4ad None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:56.909469 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e5b22d-74ef-4c25-9bb6-8467db05e4ad None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:56.909746 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e5b22d-74ef-4c25-9bb6-8467db05e4ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:56.971114 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-48e5b22d-74ef-4c25-9bb6-8467db05e4ad None None] MFA Rules not processed for user `3cd734291818490db32874612e1529ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:36:56.972339 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-48e5b22d-74ef-4c25-9bb6-8467db05e4ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:36:57.063476 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 760/1502] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:36:56 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:36:57.078262 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fc0149-f7a0-4137-9476-495321d49880 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:57.078544 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fc0149-f7a0-4137-9476-495321d49880 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:57.078824 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fc0149-f7a0-4137-9476-495321d49880 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:57.080233 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 743/1503] 10.222.0.47 () {54 vars in 964 bytes} [Wed Aug 24 06:36:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:36:57.085447 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42238c1-6885-4ad6-9058-5def068e4afd None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:57.085748 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42238c1-6885-4ad6-9058-5def068e4afd None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:57.085992 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42238c1-6885-4ad6-9058-5def068e4afd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:57.137773 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:36:57.193684 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-a42238c1-6885-4ad6-9058-5def068e4afd None None] MFA Rules not processed for user `36639dce4f97412f89d71367f650b901`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:36:57.194878 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-a42238c1-6885-4ad6-9058-5def068e4afd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:36:57.306413 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 761/1504] 10.222.0.47 () {58 vars in 1061 bytes} [Wed Aug 24 06:36:57 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:36:57.315393 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fc0149-f7a0-4137-9476-495321d49880 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:36:57.317054 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-f5fc0149-f7a0-4137-9476-495321d49880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:36:57.378040 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:36:57.436035 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fc0149-f7a0-4137-9476-495321d49880 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:36:57.518030 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c633e584-28b1-484a-b305-359875c13903 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36639dce4f97412f89d71367f650b901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:36:57.518673 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c633e584-28b1-484a-b305-359875c13903 service cinder] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:57.518953 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c633e584-28b1-484a-b305-359875c13903 service cinder] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:57.519230 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c633e584-28b1-484a-b305-359875c13903 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:57.520857 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-c633e584-28b1-484a-b305-359875c13903 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:36:57.624556 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c633e584-28b1-484a-b305-359875c13903 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:36:57.626018 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c633e584-28b1-484a-b305-359875c13903 service cinder] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:36:57.629951 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:36:57.647539 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 744/1505] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:36:57 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:36:57.694417 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cd134f-1759-4b2c-8c70-e992b5b87178 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:57.694776 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cd134f-1759-4b2c-8c70-e992b5b87178 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:57.694971 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cd134f-1759-4b2c-8c70-e992b5b87178 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:57.696358 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 762/1506] 10.222.0.47 () {52 vars in 830 bytes} [Wed Aug 24 06:36:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:36:57.700526 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c633e584-28b1-484a-b305-359875c13903 service cinder] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:36:57.708479 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-c633e584-28b1-484a-b305-359875c13903 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:36:57.824422 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c633e584-28b1-484a-b305-359875c13903 service cinder] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:36:57.955758 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f25643-f9c6-4328-b2ac-4e80c35ab517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:36:57.956391 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f25643-f9c6-4328-b2ac-4e80c35ab517 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:57.956661 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f25643-f9c6-4328-b2ac-4e80c35ab517 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:57.956927 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f25643-f9c6-4328-b2ac-4e80c35ab517 admin admin] PATH_INFO: `/v3/users/3cd734291818490db32874612e1529ed` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:57.970413 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f25643-f9c6-4328-b2ac-4e80c35ab517 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3cd734291818490db32874612e1529ed)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:36:57.971449 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: WARNING py.warnings [None req-55f25643-f9c6-4328-b2ac-4e80c35ab517 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 06:36:57.971449 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: warnings.warn(msg) Aug 24 06:36:57.971449 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: Aug 24 06:36:57.972671 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f25643-f9c6-4328-b2ac-4e80c35ab517 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:36:57.988277 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:36:58.204139 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-55f25643-f9c6-4328-b2ac-4e80c35ab517 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3cd734291818490db32874612e1529ed'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:36:58.204683 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-55f25643-f9c6-4328-b2ac-4e80c35ab517 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3cd734291818490db32874612e1529ed'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:36:58.205105 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-55f25643-f9c6-4328-b2ac-4e80c35ab517 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3cd734291818490db32874612e1529ed'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:36:58.220598 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-55f25643-f9c6-4328-b2ac-4e80c35ab517 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3cd734291818490db32874612e1529ed'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:36:58.315871 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-55f25643-f9c6-4328-b2ac-4e80c35ab517 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3cd734291818490db32874612e1529ed'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:36:58.362597 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 745/1507] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:36:57 2022] DELETE /identity/v3/users/3cd734291818490db32874612e1529ed => generated 0 bytes in 663 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:36:58.374992 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7801cb-cf3a-4f88-a293-121a1d89460f None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:58.375566 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7801cb-cf3a-4f88-a293-121a1d89460f None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:58.375677 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7801cb-cf3a-4f88-a293-121a1d89460f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:58.437863 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-8a7801cb-cf3a-4f88-a293-121a1d89460f None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:36:58.439105 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-8a7801cb-cf3a-4f88-a293-121a1d89460f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:36:58.529645 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:36:58.542417 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 763/1508] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:36:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:36:58.553160 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f25643-f9c6-4328-b2ac-4e80c35ab517 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:36:58.554437 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-55f25643-f9c6-4328-b2ac-4e80c35ab517 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:36:58.675931 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f25643-f9c6-4328-b2ac-4e80c35ab517 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:36:58.742973 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:36:58.774773 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ed1e29-b5f9-4f9a-8010-8bf703b6a5bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:36:58.774773 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ed1e29-b5f9-4f9a-8010-8bf703b6a5bb service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:58.775215 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ed1e29-b5f9-4f9a-8010-8bf703b6a5bb service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:58.775215 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ed1e29-b5f9-4f9a-8010-8bf703b6a5bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:58.776846 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-e3ed1e29-b5f9-4f9a-8010-8bf703b6a5bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:36:58.908534 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ed1e29-b5f9-4f9a-8010-8bf703b6a5bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:36:58.910197 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ed1e29-b5f9-4f9a-8010-8bf703b6a5bb service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:36:58.929769 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 746/1509] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:36:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:36:59.423072 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7801cb-cf3a-4f88-a293-121a1d89460f None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:36:59.424421 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-8a7801cb-cf3a-4f88-a293-121a1d89460f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:36:59.439314 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7801cb-cf3a-4f88-a293-121a1d89460f None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:36:59.554864 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a1ccb2-8164-46fe-8f3f-561a7af0894a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:36:59.555506 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a1ccb2-8164-46fe-8f3f-561a7af0894a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:36:59.555780 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a1ccb2-8164-46fe-8f3f-561a7af0894a admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:36:59.556058 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a1ccb2-8164-46fe-8f3f-561a7af0894a admin admin] PATH_INFO: `/v3/projects/ab456fa0620847b49b1433670a448d1f` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:36:59.567023 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a1ccb2-8164-46fe-8f3f-561a7af0894a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab456fa0620847b49b1433670a448d1f)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:36:59.568098 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: WARNING py.warnings [None req-28a1ccb2-8164-46fe-8f3f-561a7af0894a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 06:36:59.568098 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: warnings.warn(msg) Aug 24 06:36:59.568098 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: Aug 24 06:36:59.569318 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a1ccb2-8164-46fe-8f3f-561a7af0894a admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:36:59.714595 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-28a1ccb2-8164-46fe-8f3f-561a7af0894a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab456fa0620847b49b1433670a448d1f', 'request_id': 'req-28a1ccb2-8164-46fe-8f3f-561a7af0894a'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:36:59.782928 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-28a1ccb2-8164-46fe-8f3f-561a7af0894a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab456fa0620847b49b1433670a448d1f', 'request_id': 'req-28a1ccb2-8164-46fe-8f3f-561a7af0894a'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:36:59.797493 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-28a1ccb2-8164-46fe-8f3f-561a7af0894a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab456fa0620847b49b1433670a448d1f', 'request_id': 'req-28a1ccb2-8164-46fe-8f3f-561a7af0894a'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:36:59.823535 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-28a1ccb2-8164-46fe-8f3f-561a7af0894a admin admin] The token cache is being invalidate because project ab456fa0620847b49b1433670a448d1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81723) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:36:59.823819 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-28a1ccb2-8164-46fe-8f3f-561a7af0894a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:36:59.824350 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-28a1ccb2-8164-46fe-8f3f-561a7af0894a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:36:59.826244 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 764/1510] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:36:59 2022] DELETE /identity/v3/projects/ab456fa0620847b49b1433670a448d1f => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:37:08.927510 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=62 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:37:09.821535 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:38:30.516855 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8e70c3-03c3-488c-ac8e-9ed117067620 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:38:30.517701 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8e70c3-03c3-488c-ac8e-9ed117067620 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:38:30.517701 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8e70c3-03c3-488c-ac8e-9ed117067620 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:38:30.519943 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 747/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 06:38:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 06:38:30.528659 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029b9869-0c42-4104-bfb9-51afa6c5090a None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:38:30.529460 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029b9869-0c42-4104-bfb9-51afa6c5090a None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:38:30.529946 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029b9869-0c42-4104-bfb9-51afa6c5090a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:38:30.662926 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-029b9869-0c42-4104-bfb9-51afa6c5090a None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:38:30.664543 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-029b9869-0c42-4104-bfb9-51afa6c5090a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:38:30.816409 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 765/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 06:38:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:38:30.980650 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8e70c3-03c3-488c-ac8e-9ed117067620 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:38:30.988134 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-eb8e70c3-03c3-488c-ac8e-9ed117067620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:38:31.130895 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8e70c3-03c3-488c-ac8e-9ed117067620 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:38:31.137470 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e16e8e-091c-4f4b-87fd-f27e706541e8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4f49c1b08c460f8ba6c451c9c92c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:38:31.138089 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e16e8e-091c-4f4b-87fd-f27e706541e8 service nova] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:38:31.138373 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e16e8e-091c-4f4b-87fd-f27e706541e8 service nova] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:38:31.138648 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e16e8e-091c-4f4b-87fd-f27e706541e8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:38:31.147307 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e16e8e-091c-4f4b-87fd-f27e706541e8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:38:31.149243 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e16e8e-091c-4f4b-87fd-f27e706541e8 service nova] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:38:31.161622 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 748/1513] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:38:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:38:32.067281 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47804479-b8ad-4edf-a014-0ce3d0c73287 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:38:32.067881 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47804479-b8ad-4edf-a014-0ce3d0c73287 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:38:32.067881 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47804479-b8ad-4edf-a014-0ce3d0c73287 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:38:32.069258 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 766/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 06:38:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:38:32.075539 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28129cf-6d41-487e-8a39-08de5ba19d9d None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:38:32.076099 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28129cf-6d41-487e-8a39-08de5ba19d9d None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:38:32.076206 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28129cf-6d41-487e-8a39-08de5ba19d9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:38:32.097012 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-d28129cf-6d41-487e-8a39-08de5ba19d9d None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:38:32.098269 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-d28129cf-6d41-487e-8a39-08de5ba19d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:38:32.107230 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 749/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 06:38:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:38:32.267002 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47804479-b8ad-4edf-a014-0ce3d0c73287 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:38:32.289045 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47804479-b8ad-4edf-a014-0ce3d0c73287 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:38:32.297853 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b122ca12-bb4a-43a2-a20b-159cad011100 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4f49c1b08c460f8ba6c451c9c92c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:38:32.298505 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b122ca12-bb4a-43a2-a20b-159cad011100 service nova] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:38:32.298781 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b122ca12-bb4a-43a2-a20b-159cad011100 service nova] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:38:32.299162 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b122ca12-bb4a-43a2-a20b-159cad011100 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:38:32.308634 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b122ca12-bb4a-43a2-a20b-159cad011100 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:38:32.310297 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b122ca12-bb4a-43a2-a20b-159cad011100 service nova] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:38:32.320545 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 767/1516] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:38:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:38:32.830860 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a583729b-a5ed-4659-9106-036451b485e5 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:38:32.831355 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a583729b-a5ed-4659-9106-036451b485e5 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:38:32.831833 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a583729b-a5ed-4659-9106-036451b485e5 None None] PATH_INFO: `/v3` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:38:32.834206 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 750/1517] 10.222.0.67 () {54 vars in 925 bytes} [Wed Aug 24 06:38:32 2022] GET /identity/v3 => generated 255 bytes in 5 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:38:32.841478 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd6e4ca-c1a8-482d-951e-6466357c30b9 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:38:32.841764 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd6e4ca-c1a8-482d-951e-6466357c30b9 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:38:32.842050 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd6e4ca-c1a8-482d-951e-6466357c30b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:38:32.907141 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-ebd6e4ca-c1a8-482d-951e-6466357c30b9 None None] MFA Rules not processed for user `306188d42dbd4b5b84082ab3842eaf9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:38:32.908293 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-ebd6e4ca-c1a8-482d-951e-6466357c30b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:38:32.927418 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:38:33.001958 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 768/1518] 10.222.0.67 () {58 vars in 1015 bytes} [Wed Aug 24 06:38:32 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:38:37.328921 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9117d9-eb69-4560-9f23-e2ac2310f8f7 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:38:37.329503 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9117d9-eb69-4560-9f23-e2ac2310f8f7 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:38:37.329503 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9117d9-eb69-4560-9f23-e2ac2310f8f7 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:38:37.330806 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 751/1519] 10.222.0.47 () {54 vars in 961 bytes} [Wed Aug 24 06:38:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:38:37.336868 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95db7561-0e3f-4403-b05c-41491ceea395 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:38:37.337379 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95db7561-0e3f-4403-b05c-41491ceea395 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:38:37.337476 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95db7561-0e3f-4403-b05c-41491ceea395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:38:37.357546 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-95db7561-0e3f-4403-b05c-41491ceea395 None None] MFA Rules not processed for user `306188d42dbd4b5b84082ab3842eaf9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:38:37.358765 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-95db7561-0e3f-4403-b05c-41491ceea395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:38:37.367797 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 769/1520] 10.222.0.47 () {58 vars in 1058 bytes} [Wed Aug 24 06:38:37 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:38:37.376651 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9117d9-eb69-4560-9f23-e2ac2310f8f7 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:38:37.399246 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9117d9-eb69-4560-9f23-e2ac2310f8f7 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:38:37.408405 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b580f3-8615-48c9-9ffc-b1cb5dbfbdb1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '306188d42dbd4b5b84082ab3842eaf9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:38:37.409487 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b580f3-8615-48c9-9ffc-b1cb5dbfbdb1 service placement] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:38:37.409970 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b580f3-8615-48c9-9ffc-b1cb5dbfbdb1 service placement] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:38:37.410439 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b580f3-8615-48c9-9ffc-b1cb5dbfbdb1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:38:37.426689 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b580f3-8615-48c9-9ffc-b1cb5dbfbdb1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:38:37.429651 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b580f3-8615-48c9-9ffc-b1cb5dbfbdb1 service placement] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:38:37.447496 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 752/1521] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:38:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:38:43.766831 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c6721f-0876-4e87-90d7-8e77757e16ff None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:38:43.767609 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c6721f-0876-4e87-90d7-8e77757e16ff None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:38:43.767609 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c6721f-0876-4e87-90d7-8e77757e16ff None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:38:43.768755 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 770/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 06:38:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:38:43.774925 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d217e20d-c25b-4976-bd3b-91f4de1e89ba None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:38:43.775722 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d217e20d-c25b-4976-bd3b-91f4de1e89ba None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:38:43.775722 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d217e20d-c25b-4976-bd3b-91f4de1e89ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:38:43.799085 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-d217e20d-c25b-4976-bd3b-91f4de1e89ba None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:38:43.800229 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-d217e20d-c25b-4976-bd3b-91f4de1e89ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:38:43.809004 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 753/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 06:38:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:38:43.976689 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c6721f-0876-4e87-90d7-8e77757e16ff None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:38:43.997281 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c6721f-0876-4e87-90d7-8e77757e16ff None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:38:44.003231 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de173aab-bfdd-4acf-bc8d-d55e02024e41 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4f49c1b08c460f8ba6c451c9c92c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:38:44.003852 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de173aab-bfdd-4acf-bc8d-d55e02024e41 service nova] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:38:44.004130 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de173aab-bfdd-4acf-bc8d-d55e02024e41 service nova] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:38:44.004401 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de173aab-bfdd-4acf-bc8d-d55e02024e41 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:38:44.014047 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de173aab-bfdd-4acf-bc8d-d55e02024e41 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:38:44.015643 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de173aab-bfdd-4acf-bc8d-d55e02024e41 service nova] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:38:44.029777 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 771/1524] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:38:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:38:44.919666 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b365c4c2-8b4c-4950-b635-6aab132b652c None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:38:44.920211 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b365c4c2-8b4c-4950-b635-6aab132b652c None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:38:44.920211 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b365c4c2-8b4c-4950-b635-6aab132b652c None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:38:44.921622 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 754/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 06:38:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:38:44.926989 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00482f3-14dd-406d-938a-b017680e0704 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:38:44.927529 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00482f3-14dd-406d-938a-b017680e0704 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:38:44.927529 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00482f3-14dd-406d-938a-b017680e0704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:38:44.947655 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-c00482f3-14dd-406d-938a-b017680e0704 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:38:44.948965 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-c00482f3-14dd-406d-938a-b017680e0704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:38:44.957076 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 772/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 06:38:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:38:45.112951 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b365c4c2-8b4c-4950-b635-6aab132b652c None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:38:45.129427 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b365c4c2-8b4c-4950-b635-6aab132b652c None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:38:45.135065 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98948fcc-70fc-4cc9-99d1-18014979050e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4f49c1b08c460f8ba6c451c9c92c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:38:45.135753 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98948fcc-70fc-4cc9-99d1-18014979050e service nova] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:38:45.136038 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98948fcc-70fc-4cc9-99d1-18014979050e service nova] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:38:45.136318 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98948fcc-70fc-4cc9-99d1-18014979050e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:38:45.145204 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98948fcc-70fc-4cc9-99d1-18014979050e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:38:45.146802 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98948fcc-70fc-4cc9-99d1-18014979050e service nova] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:38:45.157366 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 755/1527] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:38:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:38:49.114117 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4411c9-70f3-4c6a-b49e-a92c40aaf1df None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:38:49.114684 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4411c9-70f3-4c6a-b49e-a92c40aaf1df None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:38:49.114684 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4411c9-70f3-4c6a-b49e-a92c40aaf1df None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:38:49.116034 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 773/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 06:38:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:38:49.122136 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251d1a41-01e2-47ee-b29c-94b92135cbfe None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:38:49.122678 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251d1a41-01e2-47ee-b29c-94b92135cbfe None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:38:49.122785 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251d1a41-01e2-47ee-b29c-94b92135cbfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:38:49.143661 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-251d1a41-01e2-47ee-b29c-94b92135cbfe None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:38:49.144940 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-251d1a41-01e2-47ee-b29c-94b92135cbfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:38:49.154270 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 756/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 06:38:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:38:49.442507 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4411c9-70f3-4c6a-b49e-a92c40aaf1df None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:38:49.444302 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-7c4411c9-70f3-4c6a-b49e-a92c40aaf1df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:38:49.541954 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4411c9-70f3-4c6a-b49e-a92c40aaf1df None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:38:49.546727 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56b5b88-65ee-4e21-b2a6-c7f8e3a553b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:38:49.547429 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b5b88-65ee-4e21-b2a6-c7f8e3a553b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:38:49.547756 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b5b88-65ee-4e21-b2a6-c7f8e3a553b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:38:49.548068 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b5b88-65ee-4e21-b2a6-c7f8e3a553b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:38:49.558749 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56b5b88-65ee-4e21-b2a6-c7f8e3a553b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:38:49.560191 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56b5b88-65ee-4e21-b2a6-c7f8e3a553b0 service neutron] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:38:49.570048 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 774/1530] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:38:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:38:51.507605 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28731603-9c90-4601-8c29-4bb4f2329817 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:38:51.508154 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28731603-9c90-4601-8c29-4bb4f2329817 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:38:51.508296 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28731603-9c90-4601-8c29-4bb4f2329817 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:38:51.509628 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 757/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 06:38:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 06:38:51.515459 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e6da5-17cc-4c25-a7f2-69b09813dad3 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:38:51.515899 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e6da5-17cc-4c25-a7f2-69b09813dad3 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:38:51.516011 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e6da5-17cc-4c25-a7f2-69b09813dad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:38:51.536808 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-c97e6da5-17cc-4c25-a7f2-69b09813dad3 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:38:51.538032 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-c97e6da5-17cc-4c25-a7f2-69b09813dad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:38:51.547155 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 775/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 06:38:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:38:51.836342 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28731603-9c90-4601-8c29-4bb4f2329817 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:38:51.839350 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-28731603-9c90-4601-8c29-4bb4f2329817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:38:51.854057 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28731603-9c90-4601-8c29-4bb4f2329817 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:38:51.857709 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62e28a4-d59f-4ddd-a095-1d2167d605dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:38:51.858319 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62e28a4-d59f-4ddd-a095-1d2167d605dc service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:38:51.858599 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62e28a4-d59f-4ddd-a095-1d2167d605dc service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:38:51.858883 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62e28a4-d59f-4ddd-a095-1d2167d605dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:38:51.869751 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62e28a4-d59f-4ddd-a095-1d2167d605dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:38:51.871254 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62e28a4-d59f-4ddd-a095-1d2167d605dc service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:38:51.884622 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 758/1533] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:38:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:38:55.501281 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425d3142-09c7-400c-9ec0-e542b36d67f9 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:38:55.502015 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425d3142-09c7-400c-9ec0-e542b36d67f9 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:38:55.502015 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425d3142-09c7-400c-9ec0-e542b36d67f9 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:38:55.503268 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 776/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 06:38:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:38:55.509800 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830f6f5b-0ced-45e6-a2ab-448e0ca8b2d7 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:38:55.511018 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830f6f5b-0ced-45e6-a2ab-448e0ca8b2d7 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:38:55.511018 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830f6f5b-0ced-45e6-a2ab-448e0ca8b2d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:38:55.536632 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-830f6f5b-0ced-45e6-a2ab-448e0ca8b2d7 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:38:55.538030 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-830f6f5b-0ced-45e6-a2ab-448e0ca8b2d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:38:55.547459 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 759/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 06:38:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:38:56.131259 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425d3142-09c7-400c-9ec0-e542b36d67f9 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:38:56.133734 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-425d3142-09c7-400c-9ec0-e542b36d67f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:38:56.212147 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:38:56.226263 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425d3142-09c7-400c-9ec0-e542b36d67f9 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:38:56.230513 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586023b9-c76c-4d1b-8600-c3102daffae5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36639dce4f97412f89d71367f650b901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:38:56.231131 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586023b9-c76c-4d1b-8600-c3102daffae5 service cinder] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:38:56.231410 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586023b9-c76c-4d1b-8600-c3102daffae5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:38:56.231682 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586023b9-c76c-4d1b-8600-c3102daffae5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:38:56.241760 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586023b9-c76c-4d1b-8600-c3102daffae5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:38:56.244142 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586023b9-c76c-4d1b-8600-c3102daffae5 service cinder] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:38:56.254704 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 777/1536] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:38:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:38:58.447750 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6619ab-1504-4b57-8c7e-d6c0472a4c97 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:38:58.448342 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6619ab-1504-4b57-8c7e-d6c0472a4c97 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:38:58.448594 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6619ab-1504-4b57-8c7e-d6c0472a4c97 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:38:58.451076 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 760/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 06:38:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:38:58.457684 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f66218c-6d87-49c3-a77f-c4643513f324 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:38:58.458462 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f66218c-6d87-49c3-a77f-c4643513f324 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:38:58.458462 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f66218c-6d87-49c3-a77f-c4643513f324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:38:58.479532 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-3f66218c-6d87-49c3-a77f-c4643513f324 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:38:58.480748 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-3f66218c-6d87-49c3-a77f-c4643513f324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:38:58.489339 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 778/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 06:38:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:38:59.076818 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6619ab-1504-4b57-8c7e-d6c0472a4c97 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:38:59.081704 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-4e6619ab-1504-4b57-8c7e-d6c0472a4c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:38:59.088268 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=84 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:38:59.094552 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6619ab-1504-4b57-8c7e-d6c0472a4c97 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:38:59.098262 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dc5bb6-3416-46bc-894c-574cddcd4292 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36639dce4f97412f89d71367f650b901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:38:59.098879 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dc5bb6-3416-46bc-894c-574cddcd4292 service cinder] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:38:59.099163 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dc5bb6-3416-46bc-894c-574cddcd4292 service cinder] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:38:59.099430 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dc5bb6-3416-46bc-894c-574cddcd4292 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:38:59.109954 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dc5bb6-3416-46bc-894c-574cddcd4292 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:38:59.113640 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dc5bb6-3416-46bc-894c-574cddcd4292 service cinder] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:38:59.122112 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 761/1539] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:38:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:39:08.474383 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=11 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:39:09.142495 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=5 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:40:25.057076 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a55927f-8c0d-40f4-914b-87ea5f88b020 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:25.063040 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a55927f-8c0d-40f4-914b-87ea5f88b020 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:25.063040 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a55927f-8c0d-40f4-914b-87ea5f88b020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:25.093421 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-2a55927f-8c0d-40f4-914b-87ea5f88b020 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:40:25.094671 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-2a55927f-8c0d-40f4-914b-87ea5f88b020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:25.103691 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 779/1540] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:40:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:40:25.113173 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beb6d55-f49e-468d-aaae-3d1220e7a518 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:25.114097 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beb6d55-f49e-468d-aaae-3d1220e7a518 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:25.114543 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beb6d55-f49e-468d-aaae-3d1220e7a518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:25.141647 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-2beb6d55-f49e-468d-aaae-3d1220e7a518 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:40:25.143239 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-2beb6d55-f49e-468d-aaae-3d1220e7a518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:25.155523 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 762/1541] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:40:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:40:25.161428 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a55927f-8c0d-40f4-914b-87ea5f88b020 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:25.181216 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a55927f-8c0d-40f4-914b-87ea5f88b020 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:25.186292 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af2f9b9-2408-417b-9402-3d618b28fa8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:25.186944 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af2f9b9-2408-417b-9402-3d618b28fa8f admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:25.187229 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af2f9b9-2408-417b-9402-3d618b28fa8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:25.187502 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af2f9b9-2408-417b-9402-3d618b28fa8f admin admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:25.188169 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af2f9b9-2408-417b-9402-3d618b28fa8f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:40:25.188635 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af2f9b9-2408-417b-9402-3d618b28fa8f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:25.190927 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af2f9b9-2408-417b-9402-3d618b28fa8f admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:25.211178 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 780/1542] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 24 06:40:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:40:25.219594 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beb6d55-f49e-468d-aaae-3d1220e7a518 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:25.235371 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beb6d55-f49e-468d-aaae-3d1220e7a518 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:25.240745 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c36613-3554-466c-925e-2da79790c18f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:25.241734 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c36613-3554-466c-925e-2da79790c18f admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:25.242239 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c36613-3554-466c-925e-2da79790c18f admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:25.242737 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c36613-3554-466c-925e-2da79790c18f admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:25.243590 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c36613-3554-466c-925e-2da79790c18f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:25.245757 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c36613-3554-466c-925e-2da79790c18f admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:25.294916 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111cbc4f-3d27-4eeb-aac3-d1c0ab298e51 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:25.302754 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111cbc4f-3d27-4eeb-aac3-d1c0ab298e51 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:25.302754 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111cbc4f-3d27-4eeb-aac3-d1c0ab298e51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:25.324230 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-111cbc4f-3d27-4eeb-aac3-d1c0ab298e51 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:40:25.325602 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-111cbc4f-3d27-4eeb-aac3-d1c0ab298e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:25.346771 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 781/1543] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:40:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:40:25.355906 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8497b241-a6d7-4916-8c5d-27b687a022a3 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:25.360059 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8497b241-a6d7-4916-8c5d-27b687a022a3 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:25.360059 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8497b241-a6d7-4916-8c5d-27b687a022a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:25.384958 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-8497b241-a6d7-4916-8c5d-27b687a022a3 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:40:25.386235 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-8497b241-a6d7-4916-8c5d-27b687a022a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:25.396320 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 782/1544] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:40:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:40:25.410852 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8497b241-a6d7-4916-8c5d-27b687a022a3 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:25.411859 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 763/1545] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:40:25 2022] POST /identity/v3/projects => generated 377 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:40:25.421800 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8497b241-a6d7-4916-8c5d-27b687a022a3 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:25.426311 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c36613-3554-466c-925e-2da79790c18f admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:25.530034 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc61799-ed2b-49f0-acdd-9a3804a79006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:25.530978 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc61799-ed2b-49f0-acdd-9a3804a79006 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:25.531445 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc61799-ed2b-49f0-acdd-9a3804a79006 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:25.532062 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc61799-ed2b-49f0-acdd-9a3804a79006 admin admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:25.532898 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc61799-ed2b-49f0-acdd-9a3804a79006 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:40:25.533681 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc61799-ed2b-49f0-acdd-9a3804a79006 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:25.535698 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc61799-ed2b-49f0-acdd-9a3804a79006 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:25.546926 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c36613-3554-466c-925e-2da79790c18f admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:25.559591 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1164fe0-f3f2-4e23-843c-4839f2245dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:25.560275 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 783/1546] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 24 06:40:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:40:25.560822 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1164fe0-f3f2-4e23-843c-4839f2245dca admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:25.561291 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1164fe0-f3f2-4e23-843c-4839f2245dca admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:25.561752 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe88acc-c4c2-4d9a-acfb-47c229ff6184 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:25.562236 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe88acc-c4c2-4d9a-acfb-47c229ff6184 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:25.562594 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1164fe0-f3f2-4e23-843c-4839f2245dca admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:25.562850 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe88acc-c4c2-4d9a-acfb-47c229ff6184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:25.563254 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1164fe0-f3f2-4e23-843c-4839f2245dca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:25.564345 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: WARNING py.warnings [None req-b1164fe0-f3f2-4e23-843c-4839f2245dca admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 06:40:25.564345 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: warnings.warn(msg) Aug 24 06:40:25.564345 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: Aug 24 06:40:25.566293 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1164fe0-f3f2-4e23-843c-4839f2245dca admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:25.630360 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-8fe88acc-c4c2-4d9a-acfb-47c229ff6184 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:40:25.630360 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-8fe88acc-c4c2-4d9a-acfb-47c229ff6184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:25.660803 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 784/1547] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:40:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:40:25.669029 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe88acc-c4c2-4d9a-acfb-47c229ff6184 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:25.676388 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe88acc-c4c2-4d9a-acfb-47c229ff6184 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:25.684178 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf210e8-8596-4f49-96c8-6e7f093d323b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:25.684810 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf210e8-8596-4f49-96c8-6e7f093d323b admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:25.685096 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf210e8-8596-4f49-96c8-6e7f093d323b admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:25.685403 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf210e8-8596-4f49-96c8-6e7f093d323b admin admin] PATH_INFO: `/v3/projects` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:25.686038 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf210e8-8596-4f49-96c8-6e7f093d323b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:25.687070 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: WARNING py.warnings [None req-fcf210e8-8596-4f49-96c8-6e7f093d323b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 06:40:25.687070 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: warnings.warn(msg) Aug 24 06:40:25.687070 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: Aug 24 06:40:25.688284 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf210e8-8596-4f49-96c8-6e7f093d323b admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:25.706395 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 764/1548] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:40:25 2022] POST /identity/v3/users => generated 289 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:40:25.708231 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8887e5-71a7-4592-853b-01ccd1d19ae6 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:25.713557 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8887e5-71a7-4592-853b-01ccd1d19ae6 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:25.713557 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8887e5-71a7-4592-853b-01ccd1d19ae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:25.745761 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-ae8887e5-71a7-4592-853b-01ccd1d19ae6 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:40:25.753584 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-ae8887e5-71a7-4592-853b-01ccd1d19ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:25.763047 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 765/1549] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:40:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:40:25.764319 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8887e5-71a7-4592-853b-01ccd1d19ae6 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:25.785536 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8887e5-71a7-4592-853b-01ccd1d19ae6 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:25.795874 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7def3195-85d7-41f9-a41a-28d247475b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:25.796573 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7def3195-85d7-41f9-a41a-28d247475b97 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:25.796852 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7def3195-85d7-41f9-a41a-28d247475b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:25.797125 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7def3195-85d7-41f9-a41a-28d247475b97 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:25.797899 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7def3195-85d7-41f9-a41a-28d247475b97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:25.800645 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7def3195-85d7-41f9-a41a-28d247475b97 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:25.822286 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 785/1550] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:40:25 2022] POST /identity/v3/projects => generated 357 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:40:25.828826 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf210e8-8596-4f49-96c8-6e7f093d323b admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:25.836090 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 766/1551] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:40:25 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:40:25.837628 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d8e4c5-916d-4958-bb36-0f5185511e5a None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:25.837926 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d8e4c5-916d-4958-bb36-0f5185511e5a None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:25.838205 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d8e4c5-916d-4958-bb36-0f5185511e5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:25.865254 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-c1d8e4c5-916d-4958-bb36-0f5185511e5a None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:40:25.866598 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-c1d8e4c5-916d-4958-bb36-0f5185511e5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:25.902490 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf210e8-8596-4f49-96c8-6e7f093d323b admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:25.907128 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ec9d73-76c1-4921-b970-6caf5742b9fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:25.907810 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ec9d73-76c1-4921-b970-6caf5742b9fd admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:25.908753 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ec9d73-76c1-4921-b970-6caf5742b9fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:25.909398 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ec9d73-76c1-4921-b970-6caf5742b9fd admin admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:25.909609 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ec9d73-76c1-4921-b970-6caf5742b9fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:40:25.909891 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ec9d73-76c1-4921-b970-6caf5742b9fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:25.911629 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ec9d73-76c1-4921-b970-6caf5742b9fd admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:25.924546 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 786/1552] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 24 06:40:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:40:25.930118 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ec9d73-76c1-4921-b970-6caf5742b9fd admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:25.933932 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 767/1553] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:40:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:40:25.940116 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f24b54-660c-48dd-b779-f18bd780c9c0 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:25.940116 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f24b54-660c-48dd-b779-f18bd780c9c0 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:25.940116 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f24b54-660c-48dd-b779-f18bd780c9c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:25.948881 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ec9d73-76c1-4921-b970-6caf5742b9fd admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:25.954352 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fa814b-3a10-48fc-885b-e9fdb6d19eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:25.955545 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fa814b-3a10-48fc-885b-e9fdb6d19eaf admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:25.956043 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fa814b-3a10-48fc-885b-e9fdb6d19eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:25.956532 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fa814b-3a10-48fc-885b-e9fdb6d19eaf admin admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:25.957175 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fa814b-3a10-48fc-885b-e9fdb6d19eaf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:25.957949 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-76f24b54-660c-48dd-b779-f18bd780c9c0 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:40:25.959154 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fa814b-3a10-48fc-885b-e9fdb6d19eaf admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:25.959290 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-76f24b54-660c-48dd-b779-f18bd780c9c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:25.967411 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 768/1554] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:40:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:40:25.968877 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd35820c-4a92-4212-af4a-a091a6c0325e None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:25.969013 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd35820c-4a92-4212-af4a-a091a6c0325e None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:25.969217 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd35820c-4a92-4212-af4a-a091a6c0325e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:25.989462 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-cd35820c-4a92-4212-af4a-a091a6c0325e None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:40:25.990936 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-cd35820c-4a92-4212-af4a-a091a6c0325e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:25.999018 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 769/1555] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:40:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:40:25.999891 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd35820c-4a92-4212-af4a-a091a6c0325e None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:26.009730 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd35820c-4a92-4212-af4a-a091a6c0325e None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:26.014934 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e29ca9b-79af-438a-854c-9ec956a493a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:26.015557 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e29ca9b-79af-438a-854c-9ec956a493a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:26.015871 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e29ca9b-79af-438a-854c-9ec956a493a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:26.016143 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e29ca9b-79af-438a-854c-9ec956a493a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:26.016978 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e29ca9b-79af-438a-854c-9ec956a493a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:26.019623 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e29ca9b-79af-438a-854c-9ec956a493a1 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:26.030584 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 770/1556] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:40:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:40:26.040289 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daf4bb1-d40c-49b5-b9b5-cd5942243399 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:26.040289 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daf4bb1-d40c-49b5-b9b5-cd5942243399 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:26.040289 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daf4bb1-d40c-49b5-b9b5-cd5942243399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:26.053751 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-9daf4bb1-d40c-49b5-b9b5-cd5942243399 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:40:26.055030 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-9daf4bb1-d40c-49b5-b9b5-cd5942243399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:26.063238 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 771/1557] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:40:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:40:26.064569 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf49c72-3d82-452e-8918-3f75897cf543 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:26.064864 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf49c72-3d82-452e-8918-3f75897cf543 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:26.065237 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf49c72-3d82-452e-8918-3f75897cf543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:26.074635 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 787/1558] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:40:25 2022] POST /identity/v3/users => generated 279 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:40:26.075342 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:40:26.075499 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fa814b-3a10-48fc-885b-e9fdb6d19eaf admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:26.086622 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fa814b-3a10-48fc-885b-e9fdb6d19eaf admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:26.088623 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-cdf49c72-3d82-452e-8918-3f75897cf543 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:40:26.089953 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-cdf49c72-3d82-452e-8918-3f75897cf543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:26.091499 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6da459b-e89c-4fff-84b4-08136885f8f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:26.092110 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da459b-e89c-4fff-84b4-08136885f8f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:26.092383 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da459b-e89c-4fff-84b4-08136885f8f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:26.092650 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da459b-e89c-4fff-84b4-08136885f8f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:26.093392 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6da459b-e89c-4fff-84b4-08136885f8f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:26.095283 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6da459b-e89c-4fff-84b4-08136885f8f8 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:26.099908 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 772/1559] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:40:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:40:26.101368 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e963a1-56a4-45e0-8fd6-a3b573a51087 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:26.101961 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:40:26.102095 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e963a1-56a4-45e0-8fd6-a3b573a51087 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:26.102423 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e963a1-56a4-45e0-8fd6-a3b573a51087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:26.122369 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-c2e963a1-56a4-45e0-8fd6-a3b573a51087 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:40:26.126953 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-c2e963a1-56a4-45e0-8fd6-a3b573a51087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:26.132547 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 773/1560] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:40:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:40:26.133856 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01346a73-4b31-498f-ad56-6f7adf9a7ef6 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:26.134146 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01346a73-4b31-498f-ad56-6f7adf9a7ef6 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:26.134426 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01346a73-4b31-498f-ad56-6f7adf9a7ef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:26.155729 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-01346a73-4b31-498f-ad56-6f7adf9a7ef6 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:40:26.156863 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-01346a73-4b31-498f-ad56-6f7adf9a7ef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:26.167631 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 774/1561] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:40:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:40:26.168858 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2952efce-c0bc-4467-9773-83f582ae6558 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:26.169139 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2952efce-c0bc-4467-9773-83f582ae6558 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:26.169442 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2952efce-c0bc-4467-9773-83f582ae6558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:26.188382 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-2952efce-c0bc-4467-9773-83f582ae6558 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:40:26.189688 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-2952efce-c0bc-4467-9773-83f582ae6558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:26.198308 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 775/1562] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:40:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:40:26.199106 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2952efce-c0bc-4467-9773-83f582ae6558 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:26.209250 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2952efce-c0bc-4467-9773-83f582ae6558 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:26.213889 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0c4ed7-359f-4c36-818b-bf221034ebbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:26.214561 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c4ed7-359f-4c36-818b-bf221034ebbe admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:26.214841 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c4ed7-359f-4c36-818b-bf221034ebbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:26.215115 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0c4ed7-359f-4c36-818b-bf221034ebbe admin admin] PATH_INFO: `/v3/projects/ebc642b8ee3b4946aff553abbe94c26a/users/68b76ae7f1aa41f2acf7e18790887bd5/roles/e1f9b167c51c4a388cbc9594db8a465d` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:26.230706 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0c4ed7-359f-4c36-818b-bf221034ebbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebc642b8ee3b4946aff553abbe94c26a, user_id=68b76ae7f1aa41f2acf7e18790887bd5, role_id=e1f9b167c51c4a388cbc9594db8a465d)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:26.233432 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0c4ed7-359f-4c36-818b-bf221034ebbe admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:26.239790 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 788/1563] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:40:26 2022] POST /identity/v3/projects => generated 375 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:40:26.241333 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a012079b-0f00-4112-be18-cb82fed9c2af None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:26.241443 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a012079b-0f00-4112-be18-cb82fed9c2af None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:26.241623 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a012079b-0f00-4112-be18-cb82fed9c2af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:26.261879 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-a012079b-0f00-4112-be18-cb82fed9c2af None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:40:26.263163 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-a012079b-0f00-4112-be18-cb82fed9c2af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:26.323726 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 789/1564] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:40:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:40:26.324566 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a012079b-0f00-4112-be18-cb82fed9c2af None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:26.334712 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a012079b-0f00-4112-be18-cb82fed9c2af None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:26.339524 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e63463-b7a5-4470-a1b4-ebbe68459df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:26.340106 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e63463-b7a5-4470-a1b4-ebbe68459df9 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:26.340383 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e63463-b7a5-4470-a1b4-ebbe68459df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:26.340650 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e63463-b7a5-4470-a1b4-ebbe68459df9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:26.341320 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e63463-b7a5-4470-a1b4-ebbe68459df9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:26.342953 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e63463-b7a5-4470-a1b4-ebbe68459df9 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:26.353983 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 790/1565] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:40:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:40:26.355609 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0349c83c-8e0c-46aa-a6cd-b8beef988964 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:26.355902 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0349c83c-8e0c-46aa-a6cd-b8beef988964 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:26.356179 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0349c83c-8e0c-46aa-a6cd-b8beef988964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:26.375862 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-0349c83c-8e0c-46aa-a6cd-b8beef988964 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:40:26.377626 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-0349c83c-8e0c-46aa-a6cd-b8beef988964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:26.385628 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 791/1566] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:40:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:40:26.389565 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0349c83c-8e0c-46aa-a6cd-b8beef988964 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:26.396485 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0349c83c-8e0c-46aa-a6cd-b8beef988964 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:26.401444 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc4b408-d8f0-4bb4-b12e-4720c7b5f16c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:26.402085 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc4b408-d8f0-4bb4-b12e-4720c7b5f16c admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:26.402362 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc4b408-d8f0-4bb4-b12e-4720c7b5f16c admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:26.402630 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc4b408-d8f0-4bb4-b12e-4720c7b5f16c admin admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:26.403338 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc4b408-d8f0-4bb4-b12e-4720c7b5f16c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:40:26.403697 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc4b408-d8f0-4bb4-b12e-4720c7b5f16c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:26.405449 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc4b408-d8f0-4bb4-b12e-4720c7b5f16c admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:26.406147 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 776/1567] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:40:26 2022] PUT /identity/v3/projects/ebc642b8ee3b4946aff553abbe94c26a/users/68b76ae7f1aa41f2acf7e18790887bd5/roles/e1f9b167c51c4a388cbc9594db8a465d => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:40:26.406861 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0c4ed7-359f-4c36-818b-bf221034ebbe admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:26.421285 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 792/1568] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 24 06:40:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:40:26.424696 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc4b408-d8f0-4bb4-b12e-4720c7b5f16c admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:26.433086 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc4b408-d8f0-4bb4-b12e-4720c7b5f16c admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:26.472224 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0c4ed7-359f-4c36-818b-bf221034ebbe admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:26.477502 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d66389f-3107-4206-b83a-9c4c41b33ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:26.481958 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d66389f-3107-4206-b83a-9c4c41b33ba4 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:26.481958 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d66389f-3107-4206-b83a-9c4c41b33ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:26.481958 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d66389f-3107-4206-b83a-9c4c41b33ba4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:26.481958 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d66389f-3107-4206-b83a-9c4c41b33ba4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:40:26.481958 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d66389f-3107-4206-b83a-9c4c41b33ba4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:26.482454 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: WARNING py.warnings [None req-1d66389f-3107-4206-b83a-9c4c41b33ba4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 06:40:26.482454 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: warnings.warn(msg) Aug 24 06:40:26.482454 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: Aug 24 06:40:26.482454 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d66389f-3107-4206-b83a-9c4c41b33ba4 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:26.491482 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca0e42e-31d8-48e3-b423-30673af54cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:26.492163 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca0e42e-31d8-48e3-b423-30673af54cce admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:26.492467 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca0e42e-31d8-48e3-b423-30673af54cce admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:26.492763 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca0e42e-31d8-48e3-b423-30673af54cce admin admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:26.493481 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca0e42e-31d8-48e3-b423-30673af54cce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:40:26.493778 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca0e42e-31d8-48e3-b423-30673af54cce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:26.501483 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 777/1569] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 24 06:40:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:40:26.501483 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d66389f-3107-4206-b83a-9c4c41b33ba4 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:26.502390 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca0e42e-31d8-48e3-b423-30673af54cce admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:26.508032 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d66389f-3107-4206-b83a-9c4c41b33ba4 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:26.509281 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 793/1570] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 24 06:40:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:40:26.510197 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca0e42e-31d8-48e3-b423-30673af54cce admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:26.514902 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8e9f56-7bcc-4997-ae4d-b3e2085925f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:26.515091 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8e9f56-7bcc-4997-ae4d-b3e2085925f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:26.515412 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8e9f56-7bcc-4997-ae4d-b3e2085925f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:26.515629 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8e9f56-7bcc-4997-ae4d-b3e2085925f5 admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:26.516273 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8e9f56-7bcc-4997-ae4d-b3e2085925f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:26.519391 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8e9f56-7bcc-4997-ae4d-b3e2085925f5 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:26.522753 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca0e42e-31d8-48e3-b423-30673af54cce admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:26.527937 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47c3f0c-fcfc-4deb-bc6c-5b383f55be58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:26.528819 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c3f0c-fcfc-4deb-bc6c-5b383f55be58 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:26.529049 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c3f0c-fcfc-4deb-bc6c-5b383f55be58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:26.529310 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c3f0c-fcfc-4deb-bc6c-5b383f55be58 admin admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:26.530073 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47c3f0c-fcfc-4deb-bc6c-5b383f55be58 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:40:26.530483 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47c3f0c-fcfc-4deb-bc6c-5b383f55be58 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:26.532084 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47c3f0c-fcfc-4deb-bc6c-5b383f55be58 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:26.544635 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 794/1571] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 24 06:40:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:40:26.545549 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47c3f0c-fcfc-4deb-bc6c-5b383f55be58 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:26.559326 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47c3f0c-fcfc-4deb-bc6c-5b383f55be58 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:26.565508 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f934b354-f8d2-4b52-ae2a-56da1cee1302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:26.566318 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f934b354-f8d2-4b52-ae2a-56da1cee1302 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:26.566766 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f934b354-f8d2-4b52-ae2a-56da1cee1302 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:26.567140 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f934b354-f8d2-4b52-ae2a-56da1cee1302 admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:26.568023 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f934b354-f8d2-4b52-ae2a-56da1cee1302 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:26.569786 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f934b354-f8d2-4b52-ae2a-56da1cee1302 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:26.581362 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 795/1572] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:40:26 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:40:26.587454 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f934b354-f8d2-4b52-ae2a-56da1cee1302 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:26.592466 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f934b354-f8d2-4b52-ae2a-56da1cee1302 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:26.597553 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4349551d-8fe6-4b4d-ae9b-fb59ec6e2b6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:26.598206 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4349551d-8fe6-4b4d-ae9b-fb59ec6e2b6a admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:26.598486 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4349551d-8fe6-4b4d-ae9b-fb59ec6e2b6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:26.598763 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4349551d-8fe6-4b4d-ae9b-fb59ec6e2b6a admin admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:26.599399 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4349551d-8fe6-4b4d-ae9b-fb59ec6e2b6a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:40:26.599697 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4349551d-8fe6-4b4d-ae9b-fb59ec6e2b6a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:26.601573 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4349551d-8fe6-4b4d-ae9b-fb59ec6e2b6a admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:26.615222 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 796/1573] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 24 06:40:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:40:26.616465 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8874fce-4769-4643-9d93-522af1fc5808 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:26.616753 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8874fce-4769-4643-9d93-522af1fc5808 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:26.617054 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8874fce-4769-4643-9d93-522af1fc5808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:26.637101 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-a8874fce-4769-4643-9d93-522af1fc5808 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:40:26.638334 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-a8874fce-4769-4643-9d93-522af1fc5808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:26.647557 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 797/1574] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:40:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:40:26.648544 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8874fce-4769-4643-9d93-522af1fc5808 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:26.659463 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8874fce-4769-4643-9d93-522af1fc5808 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:26.664304 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a01defb-6e02-4e5b-8f2a-1c256916c993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:26.664907 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a01defb-6e02-4e5b-8f2a-1c256916c993 admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:26.665247 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a01defb-6e02-4e5b-8f2a-1c256916c993 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:26.665544 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a01defb-6e02-4e5b-8f2a-1c256916c993 admin admin] PATH_INFO: `/v3/projects` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:26.666224 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a01defb-6e02-4e5b-8f2a-1c256916c993 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:26.666732 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 778/1575] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:40:26 2022] POST /identity/v3/users => generated 288 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:40:26.667596 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8e9f56-7bcc-4997-ae4d-b3e2085925f5 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:26.668288 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a01defb-6e02-4e5b-8f2a-1c256916c993 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:26.673979 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:40:26.678794 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8e9f56-7bcc-4997-ae4d-b3e2085925f5 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:26.683664 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67129a9-be7a-4553-8131-f73b3c49d12e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:26.684372 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67129a9-be7a-4553-8131-f73b3c49d12e admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:26.684762 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67129a9-be7a-4553-8131-f73b3c49d12e admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:26.685159 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67129a9-be7a-4553-8131-f73b3c49d12e admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:26.685926 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67129a9-be7a-4553-8131-f73b3c49d12e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:26.688141 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67129a9-be7a-4553-8131-f73b3c49d12e admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:26.840115 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 798/1576] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:40:26 2022] POST /identity/v3/projects => generated 387 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:40:26.840931 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a01defb-6e02-4e5b-8f2a-1c256916c993 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:26.904606 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 779/1577] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:40:26 2022] POST /identity/v3/projects => generated 375 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:40:26.904606 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67129a9-be7a-4553-8131-f73b3c49d12e admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:26.911583 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a01defb-6e02-4e5b-8f2a-1c256916c993 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:26.966250 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa341053-83ca-459f-a3f9-12d1da67cf65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:26.966875 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa341053-83ca-459f-a3f9-12d1da67cf65 admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:26.967154 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa341053-83ca-459f-a3f9-12d1da67cf65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:26.967430 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa341053-83ca-459f-a3f9-12d1da67cf65 admin admin] PATH_INFO: `/v3/projects` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:26.968233 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa341053-83ca-459f-a3f9-12d1da67cf65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:26.969248 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67129a9-be7a-4553-8131-f73b3c49d12e admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:26.971609 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa341053-83ca-459f-a3f9-12d1da67cf65 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:26.974970 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe57e0b-fa83-45d5-9e7a-0eeb3cd7b141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:26.975878 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe57e0b-fa83-45d5-9e7a-0eeb3cd7b141 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:26.976122 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe57e0b-fa83-45d5-9e7a-0eeb3cd7b141 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:26.976400 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe57e0b-fa83-45d5-9e7a-0eeb3cd7b141 admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:26.977099 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe57e0b-fa83-45d5-9e7a-0eeb3cd7b141 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:26.979037 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe57e0b-fa83-45d5-9e7a-0eeb3cd7b141 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:26.994935 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:40:27.115181 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 799/1578] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:40:26 2022] POST /identity/v3/projects => generated 373 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:40:27.115181 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa341053-83ca-459f-a3f9-12d1da67cf65 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:27.151704 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 780/1579] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:40:26 2022] POST /identity/v3/projects => generated 385 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:40:27.157920 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe57e0b-fa83-45d5-9e7a-0eeb3cd7b141 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:27.177714 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa341053-83ca-459f-a3f9-12d1da67cf65 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:27.198259 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:40:27.219213 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe57e0b-fa83-45d5-9e7a-0eeb3cd7b141 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:27.224360 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a7bd8e-d4a4-440e-858e-827d75486bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:27.225003 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a7bd8e-d4a4-440e-858e-827d75486bed admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:27.225311 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a7bd8e-d4a4-440e-858e-827d75486bed admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:27.225583 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a7bd8e-d4a4-440e-858e-827d75486bed admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:27.226207 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a7bd8e-d4a4-440e-858e-827d75486bed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:27.228118 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a7bd8e-d4a4-440e-858e-827d75486bed admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:27.236583 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5bc76d-fb28-4770-b801-ba9cde3274c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:27.237291 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5bc76d-fb28-4770-b801-ba9cde3274c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:27.237577 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5bc76d-fb28-4770-b801-ba9cde3274c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:27.237851 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5bc76d-fb28-4770-b801-ba9cde3274c4 admin admin] PATH_INFO: `/v3/projects/28b24aaeafe646739194e971d85444ac/users/c0dc621c3c6a41ae81ff4c43bb8f0307/roles/e1f9b167c51c4a388cbc9594db8a465d` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:27.254451 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5bc76d-fb28-4770-b801-ba9cde3274c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28b24aaeafe646739194e971d85444ac, user_id=c0dc621c3c6a41ae81ff4c43bb8f0307, role_id=e1f9b167c51c4a388cbc9594db8a465d)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:27.255770 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: WARNING py.warnings [None req-ce5bc76d-fb28-4770-b801-ba9cde3274c4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 06:40:27.255770 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: warnings.warn(msg) Aug 24 06:40:27.255770 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: Aug 24 06:40:27.257766 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5bc76d-fb28-4770-b801-ba9cde3274c4 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:27.397285 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 781/1580] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:40:27 2022] POST /identity/v3/projects => generated 387 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:40:27.398370 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a7bd8e-d4a4-440e-858e-827d75486bed admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:27.437621 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 800/1581] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:40:27 2022] PUT /identity/v3/projects/28b24aaeafe646739194e971d85444ac/users/c0dc621c3c6a41ae81ff4c43bb8f0307/roles/e1f9b167c51c4a388cbc9594db8a465d => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:40:27.437621 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5bc76d-fb28-4770-b801-ba9cde3274c4 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:27.470162 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a7bd8e-d4a4-440e-858e-827d75486bed admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:27.510130 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5bc76d-fb28-4770-b801-ba9cde3274c4 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:27.515386 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1e4872-96cd-46a5-9cb1-a69fbe931b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:27.516102 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1e4872-96cd-46a5-9cb1-a69fbe931b12 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:27.516422 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1e4872-96cd-46a5-9cb1-a69fbe931b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:27.516721 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1e4872-96cd-46a5-9cb1-a69fbe931b12 admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:27.517546 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1e4872-96cd-46a5-9cb1-a69fbe931b12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:27.519241 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1e4872-96cd-46a5-9cb1-a69fbe931b12 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:27.528644 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a47ea0-b8e8-43e5-868b-ab9dd9fe7599 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:27.529324 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a47ea0-b8e8-43e5-868b-ab9dd9fe7599 service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:27.529634 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a47ea0-b8e8-43e5-868b-ab9dd9fe7599 service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:27.529922 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a47ea0-b8e8-43e5-868b-ab9dd9fe7599 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:27.533220 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 801/1582] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:40:27 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:40:27.534388 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1e4872-96cd-46a5-9cb1-a69fbe931b12 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:27.540798 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a47ea0-b8e8-43e5-868b-ab9dd9fe7599 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:27.542328 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a47ea0-b8e8-43e5-868b-ab9dd9fe7599 service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:27.545334 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1e4872-96cd-46a5-9cb1-a69fbe931b12 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:27.551467 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c17f0c7-b82f-4191-b1c1-8740aea3a329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:27.552163 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c17f0c7-b82f-4191-b1c1-8740aea3a329 admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:27.552451 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c17f0c7-b82f-4191-b1c1-8740aea3a329 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:27.552732 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c17f0c7-b82f-4191-b1c1-8740aea3a329 admin admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:27.553409 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c17f0c7-b82f-4191-b1c1-8740aea3a329 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:27.553589 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 782/1583] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:40:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:40:27.554473 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a47ea0-b8e8-43e5-868b-ab9dd9fe7599 service neutron] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:27.555917 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c17f0c7-b82f-4191-b1c1-8740aea3a329 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:27.566635 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a47ea0-b8e8-43e5-868b-ab9dd9fe7599 service neutron] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:27.572505 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1d2a22-4dc6-4ad6-a9a4-2cde7ad3d1a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:27.573409 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1d2a22-4dc6-4ad6-a9a4-2cde7ad3d1a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:27.573713 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1d2a22-4dc6-4ad6-a9a4-2cde7ad3d1a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:27.574016 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1d2a22-4dc6-4ad6-a9a4-2cde7ad3d1a9 admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:27.574683 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1d2a22-4dc6-4ad6-a9a4-2cde7ad3d1a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:27.576866 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1d2a22-4dc6-4ad6-a9a4-2cde7ad3d1a9 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:27.652482 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 802/1584] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:40:27 2022] POST /identity/v3/users => generated 294 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:40:27.653385 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c17f0c7-b82f-4191-b1c1-8740aea3a329 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:27.664215 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c17f0c7-b82f-4191-b1c1-8740aea3a329 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:27.669293 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061842c1-feaa-4ff3-9560-3c1d1ef0895f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:27.669932 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061842c1-feaa-4ff3-9560-3c1d1ef0895f admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:27.670218 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061842c1-feaa-4ff3-9560-3c1d1ef0895f admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:27.670498 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061842c1-feaa-4ff3-9560-3c1d1ef0895f admin admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:27.671149 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061842c1-feaa-4ff3-9560-3c1d1ef0895f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:27.673158 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061842c1-feaa-4ff3-9560-3c1d1ef0895f admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:27.691913 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 783/1585] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:40:27 2022] POST /identity/v3/users => generated 288 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:40:27.692729 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1d2a22-4dc6-4ad6-a9a4-2cde7ad3d1a9 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:27.704206 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1d2a22-4dc6-4ad6-a9a4-2cde7ad3d1a9 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:27.710141 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b26c545-b117-4f37-8ed0-e8da23ab4e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:27.710874 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b26c545-b117-4f37-8ed0-e8da23ab4e94 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:27.711299 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b26c545-b117-4f37-8ed0-e8da23ab4e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:27.711640 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b26c545-b117-4f37-8ed0-e8da23ab4e94 admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:27.712329 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b26c545-b117-4f37-8ed0-e8da23ab4e94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:27.714388 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b26c545-b117-4f37-8ed0-e8da23ab4e94 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:27.823701 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 803/1586] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:40:27 2022] POST /identity/v3/users => generated 287 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:40:27.824101 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 784/1587] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:40:27 2022] POST /identity/v3/users => generated 293 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:40:27.825854 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8cb012-9d0e-44ea-aff0-703abaf6977c None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:27.826077 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8cb012-9d0e-44ea-aff0-703abaf6977c None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:27.826330 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8cb012-9d0e-44ea-aff0-703abaf6977c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:27.828040 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061842c1-feaa-4ff3-9560-3c1d1ef0895f admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:27.834155 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:40:27.839165 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061842c1-feaa-4ff3-9560-3c1d1ef0895f admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:27.846087 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb355a4-d1c1-49dd-a315-75e625367572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:27.846706 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb355a4-d1c1-49dd-a315-75e625367572 admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:27.846983 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb355a4-d1c1-49dd-a315-75e625367572 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:27.847316 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb355a4-d1c1-49dd-a315-75e625367572 admin admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:27.848113 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb355a4-d1c1-49dd-a315-75e625367572 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:27.850349 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb355a4-d1c1-49dd-a315-75e625367572 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:27.859003 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-fa8cb012-9d0e-44ea-aff0-703abaf6977c None None] MFA Rules not processed for user `c0dc621c3c6a41ae81ff4c43bb8f0307`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:40:27.860300 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-fa8cb012-9d0e-44ea-aff0-703abaf6977c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:27.936744 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 785/1588] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:40:27 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:40:27.937938 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8cb012-9d0e-44ea-aff0-703abaf6977c None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:27.952051 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8cb012-9d0e-44ea-aff0-703abaf6977c None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:27.957974 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55de6849-6bc6-405b-9948-377dc5b71834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:27.958646 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55de6849-6bc6-405b-9948-377dc5b71834 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:27.958929 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55de6849-6bc6-405b-9948-377dc5b71834 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:27.959274 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55de6849-6bc6-405b-9948-377dc5b71834 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:27.959949 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55de6849-6bc6-405b-9948-377dc5b71834 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:27.961650 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55de6849-6bc6-405b-9948-377dc5b71834 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:27.973500 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 786/1589] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:40:27 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:40:27.974375 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55de6849-6bc6-405b-9948-377dc5b71834 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:27.990268 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55de6849-6bc6-405b-9948-377dc5b71834 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:27.995251 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a63c56f-ad01-49f4-a26a-778b71ab450a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:27.999494 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a63c56f-ad01-49f4-a26a-778b71ab450a admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:27.999494 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a63c56f-ad01-49f4-a26a-778b71ab450a admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:27.999494 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a63c56f-ad01-49f4-a26a-778b71ab450a admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:27.999494 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a63c56f-ad01-49f4-a26a-778b71ab450a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:27.999494 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a63c56f-ad01-49f4-a26a-778b71ab450a admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:28.012425 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 787/1590] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:40:27 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:40:28.012425 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a63c56f-ad01-49f4-a26a-778b71ab450a admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:28.025334 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a63c56f-ad01-49f4-a26a-778b71ab450a admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:28.030130 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d52b81c-babb-4b47-a3e4-ff4ac73507a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:28.030765 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d52b81c-babb-4b47-a3e4-ff4ac73507a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:28.031841 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d52b81c-babb-4b47-a3e4-ff4ac73507a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:28.032103 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d52b81c-babb-4b47-a3e4-ff4ac73507a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:28.032880 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d52b81c-babb-4b47-a3e4-ff4ac73507a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:28.034591 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d52b81c-babb-4b47-a3e4-ff4ac73507a1 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:28.051899 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 788/1591] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:40:28 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:40:28.051899 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d52b81c-babb-4b47-a3e4-ff4ac73507a1 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:28.062046 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d52b81c-babb-4b47-a3e4-ff4ac73507a1 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:28.076597 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c888ad-ac69-421f-b3ad-513b12d375a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:28.076597 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c888ad-ac69-421f-b3ad-513b12d375a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:28.076597 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c888ad-ac69-421f-b3ad-513b12d375a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:28.076597 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c888ad-ac69-421f-b3ad-513b12d375a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:28.077100 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c888ad-ac69-421f-b3ad-513b12d375a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:28.077100 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c888ad-ac69-421f-b3ad-513b12d375a4 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:28.083549 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 789/1592] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:40:28 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:40:28.084593 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c888ad-ac69-421f-b3ad-513b12d375a4 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:28.095092 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c888ad-ac69-421f-b3ad-513b12d375a4 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:28.100861 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6028af-992b-4d70-83fd-1172ea5383a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:28.113959 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6028af-992b-4d70-83fd-1172ea5383a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:28.113959 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6028af-992b-4d70-83fd-1172ea5383a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:28.113959 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6028af-992b-4d70-83fd-1172ea5383a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:28.113959 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6028af-992b-4d70-83fd-1172ea5383a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:28.114351 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 804/1593] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:40:27 2022] POST /identity/v3/users => generated 294 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:40:28.114469 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6028af-992b-4d70-83fd-1172ea5383a3 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:28.114634 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb355a4-d1c1-49dd-a315-75e625367572 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:28.120301 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb355a4-d1c1-49dd-a315-75e625367572 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:28.125047 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 790/1594] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:40:28 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:40:28.125047 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6028af-992b-4d70-83fd-1172ea5383a3 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:28.125988 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699dfbfc-8a86-46c4-9ccc-5f3bf9366d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:28.127292 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699dfbfc-8a86-46c4-9ccc-5f3bf9366d8d admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:28.127484 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699dfbfc-8a86-46c4-9ccc-5f3bf9366d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:28.127836 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699dfbfc-8a86-46c4-9ccc-5f3bf9366d8d admin admin] PATH_INFO: `/v3/projects` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:28.128380 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699dfbfc-8a86-46c4-9ccc-5f3bf9366d8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:28.130410 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699dfbfc-8a86-46c4-9ccc-5f3bf9366d8d admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:28.134658 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6028af-992b-4d70-83fd-1172ea5383a3 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:28.139673 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657dde76-5445-4845-9ff0-e34efe9bdeb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:28.140366 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657dde76-5445-4845-9ff0-e34efe9bdeb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:28.141252 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657dde76-5445-4845-9ff0-e34efe9bdeb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:28.141252 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657dde76-5445-4845-9ff0-e34efe9bdeb1 admin admin] PATH_INFO: `/v3/projects/738113e0ae1c4cfa942acc615e748a9b/users/a4fddb5f283940838ab1e60ec8c64a9c/roles/e1f9b167c51c4a388cbc9594db8a465d` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:28.158887 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657dde76-5445-4845-9ff0-e34efe9bdeb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=738113e0ae1c4cfa942acc615e748a9b, user_id=a4fddb5f283940838ab1e60ec8c64a9c, role_id=e1f9b167c51c4a388cbc9594db8a465d)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:28.161701 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657dde76-5445-4845-9ff0-e34efe9bdeb1 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:28.251537 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 805/1595] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:40:28 2022] POST /identity/v3/projects => generated 357 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:40:28.251537 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699dfbfc-8a86-46c4-9ccc-5f3bf9366d8d admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:28.297353 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=89,keystone:INSERT=11 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:40:28.325017 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699dfbfc-8a86-46c4-9ccc-5f3bf9366d8d admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:28.330702 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b799b2-1259-4491-a1e9-e35ad5cee96f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:28.331645 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b799b2-1259-4491-a1e9-e35ad5cee96f admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:28.331926 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b799b2-1259-4491-a1e9-e35ad5cee96f admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:28.332198 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b799b2-1259-4491-a1e9-e35ad5cee96f admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:28.332855 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b799b2-1259-4491-a1e9-e35ad5cee96f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:28.334688 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b799b2-1259-4491-a1e9-e35ad5cee96f admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:28.346596 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 806/1596] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:40:28 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:40:28.348317 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b799b2-1259-4491-a1e9-e35ad5cee96f admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:28.355619 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 791/1597] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:40:28 2022] PUT /identity/v3/projects/738113e0ae1c4cfa942acc615e748a9b/users/a4fddb5f283940838ab1e60ec8c64a9c/roles/e1f9b167c51c4a388cbc9594db8a465d => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:40:28.357659 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657dde76-5445-4845-9ff0-e34efe9bdeb1 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:28.376271 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b799b2-1259-4491-a1e9-e35ad5cee96f admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:28.430262 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657dde76-5445-4845-9ff0-e34efe9bdeb1 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:28.435287 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e1e5c8-34b4-401d-aeb4-f36ff673814f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:28.435995 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e1e5c8-34b4-401d-aeb4-f36ff673814f admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:28.436279 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e1e5c8-34b4-401d-aeb4-f36ff673814f admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:28.436476 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6788e15a-68a6-40e3-9105-31ab13dca14c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:28.436604 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e1e5c8-34b4-401d-aeb4-f36ff673814f admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:28.437156 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788e15a-68a6-40e3-9105-31ab13dca14c admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:28.437282 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e1e5c8-34b4-401d-aeb4-f36ff673814f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:28.437465 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788e15a-68a6-40e3-9105-31ab13dca14c admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:28.437739 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788e15a-68a6-40e3-9105-31ab13dca14c admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:28.438390 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6788e15a-68a6-40e3-9105-31ab13dca14c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:28.438901 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e1e5c8-34b4-401d-aeb4-f36ff673814f admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:28.440541 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6788e15a-68a6-40e3-9105-31ab13dca14c admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:28.450927 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 792/1598] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:40:28 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:40:28.451827 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e1e5c8-34b4-401d-aeb4-f36ff673814f admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:28.452152 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 807/1599] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:40:28 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:40:28.453011 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6788e15a-68a6-40e3-9105-31ab13dca14c admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:28.465877 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6788e15a-68a6-40e3-9105-31ab13dca14c admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:28.471707 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e1e5c8-34b4-401d-aeb4-f36ff673814f admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:28.471983 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1957dcf1-f5e6-4b68-9465-acf9f16a8a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:28.471983 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1957dcf1-f5e6-4b68-9465-acf9f16a8a43 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:28.472161 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0213530d-bf91-4586-b629-aa311e6718b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:28.472251 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1957dcf1-f5e6-4b68-9465-acf9f16a8a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:28.472343 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1957dcf1-f5e6-4b68-9465-acf9f16a8a43 admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:28.472631 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0213530d-bf91-4586-b629-aa311e6718b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:28.472922 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0213530d-bf91-4586-b629-aa311e6718b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:28.473030 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1957dcf1-f5e6-4b68-9465-acf9f16a8a43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:28.473239 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0213530d-bf91-4586-b629-aa311e6718b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:28.474036 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0213530d-bf91-4586-b629-aa311e6718b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:28.474774 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1957dcf1-f5e6-4b68-9465-acf9f16a8a43 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:28.475535 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0213530d-bf91-4586-b629-aa311e6718b6 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:28.478059 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:40:28.487939 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 793/1600] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:40:28 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:40:28.489485 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0213530d-bf91-4586-b629-aa311e6718b6 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:28.490094 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 808/1601] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:40:28 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:40:28.491223 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1957dcf1-f5e6-4b68-9465-acf9f16a8a43 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:28.502149 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0213530d-bf91-4586-b629-aa311e6718b6 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:28.505403 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1957dcf1-f5e6-4b68-9465-acf9f16a8a43 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:28.508466 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b5a007-e570-4a43-97aa-e0563fa173e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:28.509102 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b5a007-e570-4a43-97aa-e0563fa173e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:28.509420 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b5a007-e570-4a43-97aa-e0563fa173e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:28.509912 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b5a007-e570-4a43-97aa-e0563fa173e2 admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:28.510330 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b5a007-e570-4a43-97aa-e0563fa173e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:28.510700 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f2089c-843b-41c7-b075-acbd34901976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:28.511729 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f2089c-843b-41c7-b075-acbd34901976 admin admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:28.512010 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f2089c-843b-41c7-b075-acbd34901976 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:28.512277 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b5a007-e570-4a43-97aa-e0563fa173e2 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:28.512391 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f2089c-843b-41c7-b075-acbd34901976 admin admin] PATH_INFO: `/v3/projects/e1bf995c8c544dc29f42e53866cc6bd2/users/52f796d5013d4a31b107fb86c5523c9c/roles/e1f9b167c51c4a388cbc9594db8a465d` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:28.530739 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f2089c-843b-41c7-b075-acbd34901976 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1bf995c8c544dc29f42e53866cc6bd2, user_id=52f796d5013d4a31b107fb86c5523c9c, role_id=e1f9b167c51c4a388cbc9594db8a465d)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:28.533515 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f2089c-843b-41c7-b075-acbd34901976 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:28.635807 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 809/1602] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:40:28 2022] PUT /identity/v3/projects/e1bf995c8c544dc29f42e53866cc6bd2/users/52f796d5013d4a31b107fb86c5523c9c/roles/e1f9b167c51c4a388cbc9594db8a465d => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:40:28.636744 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 794/1603] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:40:28 2022] POST /identity/v3/users => generated 285 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:40:28.637668 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b5a007-e570-4a43-97aa-e0563fa173e2 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:28.645595 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7f6793-50ba-4d88-be13-bf490820666c None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:28.645936 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7f6793-50ba-4d88-be13-bf490820666c None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:28.646173 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7f6793-50ba-4d88-be13-bf490820666c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:28.650414 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b5a007-e570-4a43-97aa-e0563fa173e2 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:28.677373 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-bb7f6793-50ba-4d88-be13-bf490820666c None None] MFA Rules not processed for user `a4fddb5f283940838ab1e60ec8c64a9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:40:28.678709 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-bb7f6793-50ba-4d88-be13-bf490820666c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:28.715321 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5d04a3-aaa6-4f5f-b1c2-33ece92ebb43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:28.715952 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d04a3-aaa6-4f5f-b1c2-33ece92ebb43 admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:28.716407 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d04a3-aaa6-4f5f-b1c2-33ece92ebb43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:28.716803 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d04a3-aaa6-4f5f-b1c2-33ece92ebb43 admin admin] PATH_INFO: `/v3/projects/521a4e5109084dd2a7fb3fb73aa202af/users/4125b9dd74a842bca7230e28171b226b/roles/e1f9b167c51c4a388cbc9594db8a465d` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:28.738048 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5d04a3-aaa6-4f5f-b1c2-33ece92ebb43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=521a4e5109084dd2a7fb3fb73aa202af, user_id=4125b9dd74a842bca7230e28171b226b, role_id=e1f9b167c51c4a388cbc9594db8a465d)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:28.741210 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5d04a3-aaa6-4f5f-b1c2-33ece92ebb43 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:28.753364 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 810/1604] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:40:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:40:28.754342 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7f6793-50ba-4d88-be13-bf490820666c None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:28.765261 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7f6793-50ba-4d88-be13-bf490820666c None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:28.770155 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e3eab7-3ef4-45ed-a7df-906e1666f12f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:28.770949 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e3eab7-3ef4-45ed-a7df-906e1666f12f admin admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:28.771231 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e3eab7-3ef4-45ed-a7df-906e1666f12f admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:28.771509 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e3eab7-3ef4-45ed-a7df-906e1666f12f admin admin] PATH_INFO: `/v3/projects/096ae1367ad84c7aaa0b5e96e22e9e32/users/8b9b29b53a7b490da29fbf6e5451c362/roles/e1f9b167c51c4a388cbc9594db8a465d` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:28.791465 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:40:28.794715 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e3eab7-3ef4-45ed-a7df-906e1666f12f admin admin] RBAC: Authorizing `identity:create_grant(project_id=096ae1367ad84c7aaa0b5e96e22e9e32, user_id=8b9b29b53a7b490da29fbf6e5451c362, role_id=e1f9b167c51c4a388cbc9594db8a465d)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:28.803915 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e3eab7-3ef4-45ed-a7df-906e1666f12f admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:28.898264 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 795/1605] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:40:28 2022] PUT /identity/v3/projects/521a4e5109084dd2a7fb3fb73aa202af/users/4125b9dd74a842bca7230e28171b226b/roles/e1f9b167c51c4a388cbc9594db8a465d => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:40:28.899132 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5d04a3-aaa6-4f5f-b1c2-33ece92ebb43 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:28.965692 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5d04a3-aaa6-4f5f-b1c2-33ece92ebb43 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:28.970852 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 811/1606] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:40:28 2022] PUT /identity/v3/projects/096ae1367ad84c7aaa0b5e96e22e9e32/users/8b9b29b53a7b490da29fbf6e5451c362/roles/e1f9b167c51c4a388cbc9594db8a465d => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:40:28.972216 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e3eab7-3ef4-45ed-a7df-906e1666f12f admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:29.039233 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ca7445-67eb-4329-8769-20f761617e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:29.040276 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ca7445-67eb-4329-8769-20f761617e21 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:29.040276 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ca7445-67eb-4329-8769-20f761617e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:29.040706 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ca7445-67eb-4329-8769-20f761617e21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:29.041470 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ca7445-67eb-4329-8769-20f761617e21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:29.043505 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ca7445-67eb-4329-8769-20f761617e21 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:29.051483 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e3eab7-3ef4-45ed-a7df-906e1666f12f admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:29.055417 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 796/1607] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:40:28 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:40:29.056743 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ca7445-67eb-4329-8769-20f761617e21 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:29.060206 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87926426-8d6f-4a94-8d35-4581865cae62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:29.060996 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87926426-8d6f-4a94-8d35-4581865cae62 admin admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:29.061371 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87926426-8d6f-4a94-8d35-4581865cae62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:29.061716 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87926426-8d6f-4a94-8d35-4581865cae62 admin admin] PATH_INFO: `/v3/projects/b91004b0497d487ea8f1f454c03bbca0/users/02ad6cdc4ffe4917a7bbf6814297f980/roles/e1f9b167c51c4a388cbc9594db8a465d` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:29.067968 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ca7445-67eb-4329-8769-20f761617e21 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:29.072692 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1709c9de-9078-445c-930c-416021ee2800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:29.073356 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1709c9de-9078-445c-930c-416021ee2800 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:29.073634 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1709c9de-9078-445c-930c-416021ee2800 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:29.073904 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1709c9de-9078-445c-930c-416021ee2800 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:29.074541 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1709c9de-9078-445c-930c-416021ee2800 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:29.076416 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1709c9de-9078-445c-930c-416021ee2800 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:29.077856 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:40:29.078738 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87926426-8d6f-4a94-8d35-4581865cae62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b91004b0497d487ea8f1f454c03bbca0, user_id=02ad6cdc4ffe4917a7bbf6814297f980, role_id=e1f9b167c51c4a388cbc9594db8a465d)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:29.082201 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87926426-8d6f-4a94-8d35-4581865cae62 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:29.088020 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 797/1608] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:40:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:40:29.089436 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c270ac-2c3b-4fcd-a502-39244b263454 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:29.089858 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c270ac-2c3b-4fcd-a502-39244b263454 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:29.090513 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c270ac-2c3b-4fcd-a502-39244b263454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:29.111630 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-a6c270ac-2c3b-4fcd-a502-39244b263454 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:40:29.112727 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-a6c270ac-2c3b-4fcd-a502-39244b263454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:29.120580 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 798/1609] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:40:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:40:29.121965 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c9b2c9-6409-43f0-abbf-866180390d32 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:29.122301 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c9b2c9-6409-43f0-abbf-866180390d32 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:29.122581 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c9b2c9-6409-43f0-abbf-866180390d32 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:29.124172 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 799/1610] 10.222.0.47 () {54 vars in 962 bytes} [Wed Aug 24 06:40:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:40:29.125240 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245dffdf-60a2-47b0-a852-27880bd51546 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:29.125525 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245dffdf-60a2-47b0-a852-27880bd51546 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:29.125800 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245dffdf-60a2-47b0-a852-27880bd51546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:29.153131 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-245dffdf-60a2-47b0-a852-27880bd51546 None None] MFA Rules not processed for user `4125b9dd74a842bca7230e28171b226b`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:40:29.154430 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-245dffdf-60a2-47b0-a852-27880bd51546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:29.219486 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 800/1611] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:40:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:40:29.220966 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097cd13d-117b-4cee-b190-6e5de4e5d2e0 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:29.221507 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097cd13d-117b-4cee-b190-6e5de4e5d2e0 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:29.235171 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097cd13d-117b-4cee-b190-6e5de4e5d2e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:29.241602 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-097cd13d-117b-4cee-b190-6e5de4e5d2e0 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:40:29.242869 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-097cd13d-117b-4cee-b190-6e5de4e5d2e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:29.254250 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 801/1612] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:40:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:40:29.254250 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097cd13d-117b-4cee-b190-6e5de4e5d2e0 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:29.263742 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097cd13d-117b-4cee-b190-6e5de4e5d2e0 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:29.269848 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bb1139-6e17-43de-bd6f-a7ea005b98c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:29.270523 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bb1139-6e17-43de-bd6f-a7ea005b98c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:29.270890 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bb1139-6e17-43de-bd6f-a7ea005b98c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:29.271141 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bb1139-6e17-43de-bd6f-a7ea005b98c6 admin admin] PATH_INFO: `/v3/projects/2d2ff0314b9846f9af2f6a72f8d0e5d5/users/d787466a655045c08ebe94eb920088a9/roles/e1f9b167c51c4a388cbc9594db8a465d` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:29.278729 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 812/1613] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:40:28 2022] PUT /identity/v3/projects/b91004b0497d487ea8f1f454c03bbca0/users/02ad6cdc4ffe4917a7bbf6814297f980/roles/e1f9b167c51c4a388cbc9594db8a465d => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:40:29.279561 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87926426-8d6f-4a94-8d35-4581865cae62 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:29.288162 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bb1139-6e17-43de-bd6f-a7ea005b98c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d2ff0314b9846f9af2f6a72f8d0e5d5, user_id=d787466a655045c08ebe94eb920088a9, role_id=e1f9b167c51c4a388cbc9594db8a465d)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:29.290865 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bb1139-6e17-43de-bd6f-a7ea005b98c6 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:29.344540 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87926426-8d6f-4a94-8d35-4581865cae62 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:29.349974 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcda00b-696b-44ad-ae34-3be70f7f1d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:29.350642 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcda00b-696b-44ad-ae34-3be70f7f1d28 admin admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:29.350927 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcda00b-696b-44ad-ae34-3be70f7f1d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:29.351203 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcda00b-696b-44ad-ae34-3be70f7f1d28 admin admin] PATH_INFO: `/v3/projects/fa25011931e84a71bd1b20a9f3766f54/users/5919707e716e4ef28b5fe800c29167cc/roles/486144ad1bcd417e86acdbc1d14327bf` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:29.366416 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcda00b-696b-44ad-ae34-3be70f7f1d28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa25011931e84a71bd1b20a9f3766f54, user_id=5919707e716e4ef28b5fe800c29167cc, role_id=486144ad1bcd417e86acdbc1d14327bf)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:29.369076 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcda00b-696b-44ad-ae34-3be70f7f1d28 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:29.444103 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 802/1614] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:40:29 2022] PUT /identity/v3/projects/2d2ff0314b9846f9af2f6a72f8d0e5d5/users/d787466a655045c08ebe94eb920088a9/roles/e1f9b167c51c4a388cbc9594db8a465d => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:40:29.445542 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12905129-6dd8-47f9-bf97-e93dd36c2857 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:29.445827 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12905129-6dd8-47f9-bf97-e93dd36c2857 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:29.446099 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12905129-6dd8-47f9-bf97-e93dd36c2857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:29.488003 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-12905129-6dd8-47f9-bf97-e93dd36c2857 None None] MFA Rules not processed for user `1f4f49c1b08c460f8ba6c451c9c92c2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:40:29.489242 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-12905129-6dd8-47f9-bf97-e93dd36c2857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:29.557126 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 813/1615] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:40:29 2022] PUT /identity/v3/projects/fa25011931e84a71bd1b20a9f3766f54/users/5919707e716e4ef28b5fe800c29167cc/roles/486144ad1bcd417e86acdbc1d14327bf => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:40:29.558727 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcda00b-696b-44ad-ae34-3be70f7f1d28 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:29.562629 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 803/1616] 10.222.0.47 () {58 vars in 1059 bytes} [Wed Aug 24 06:40:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:40:29.564122 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70056ab7-cd8b-4724-b8e3-71bcf3857267 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:29.564548 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70056ab7-cd8b-4724-b8e3-71bcf3857267 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:29.565091 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70056ab7-cd8b-4724-b8e3-71bcf3857267 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:29.566701 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 804/1617] 10.222.0.47 () {54 vars in 962 bytes} [Wed Aug 24 06:40:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:40:29.567460 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70056ab7-cd8b-4724-b8e3-71bcf3857267 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:29.641889 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcda00b-696b-44ad-ae34-3be70f7f1d28 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:29.648226 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647db0f0-b275-4796-8577-d24876ca9606 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:29.648844 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647db0f0-b275-4796-8577-d24876ca9606 service neutron] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:29.649546 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647db0f0-b275-4796-8577-d24876ca9606 service neutron] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:29.649546 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647db0f0-b275-4796-8577-d24876ca9606 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:29.658609 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70056ab7-cd8b-4724-b8e3-71bcf3857267 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:29.662819 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90b250e-1a24-4653-a9e0-d79cdfef1e29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:29.663447 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90b250e-1a24-4653-a9e0-d79cdfef1e29 service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:29.663747 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90b250e-1a24-4653-a9e0-d79cdfef1e29 service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:29.664009 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90b250e-1a24-4653-a9e0-d79cdfef1e29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:29.725836 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647db0f0-b275-4796-8577-d24876ca9606 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:29.727487 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647db0f0-b275-4796-8577-d24876ca9606 service neutron] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:29.737088 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90b250e-1a24-4653-a9e0-d79cdfef1e29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:29.738893 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 814/1618] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:40:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:40:29.739216 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90b250e-1a24-4653-a9e0-d79cdfef1e29 service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:29.740228 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8ff089-5bff-4715-bece-fa7849490f99 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:29.740519 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8ff089-5bff-4715-bece-fa7849490f99 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:29.740798 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8ff089-5bff-4715-bece-fa7849490f99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:29.762086 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-fc8ff089-5bff-4715-bece-fa7849490f99 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:40:29.762514 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 805/1619] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:40:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:40:29.763320 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-fc8ff089-5bff-4715-bece-fa7849490f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:29.766430 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8a7908-3803-4258-9371-0ccca866f760 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:29.767787 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8a7908-3803-4258-9371-0ccca866f760 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:29.767787 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8a7908-3803-4258-9371-0ccca866f760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:29.777258 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 815/1620] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:40:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:40:29.777258 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8ff089-5bff-4715-bece-fa7849490f99 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:29.789280 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8ff089-5bff-4715-bece-fa7849490f99 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:29.794874 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3089cb11-022a-4bc3-add8-a1e5fd93348b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:29.796199 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3089cb11-022a-4bc3-add8-a1e5fd93348b admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:29.796199 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3089cb11-022a-4bc3-add8-a1e5fd93348b admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:29.796492 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3089cb11-022a-4bc3-add8-a1e5fd93348b admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:29.797206 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3089cb11-022a-4bc3-add8-a1e5fd93348b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:29.799067 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3089cb11-022a-4bc3-add8-a1e5fd93348b admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:29.808577 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-ce8a7908-3803-4258-9371-0ccca866f760 None None] MFA Rules not processed for user `d787466a655045c08ebe94eb920088a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:40:29.831190 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-ce8a7908-3803-4258-9371-0ccca866f760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:29.831354 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 816/1621] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:40:29 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:40:29.831354 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b205671e-ecee-48fc-a7a1-df993a1790c8 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:29.831354 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b205671e-ecee-48fc-a7a1-df993a1790c8 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:29.831354 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b205671e-ecee-48fc-a7a1-df993a1790c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:29.837944 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-b205671e-ecee-48fc-a7a1-df993a1790c8 None None] MFA Rules not processed for user `1f4f49c1b08c460f8ba6c451c9c92c2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:40:29.839335 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-b205671e-ecee-48fc-a7a1-df993a1790c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:29.886604 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 806/1622] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:40:29 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:40:29.887874 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8a7908-3803-4258-9371-0ccca866f760 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:29.933438 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 817/1623] 10.222.0.47 () {58 vars in 1059 bytes} [Wed Aug 24 06:40:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:40:29.934891 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b205671e-ecee-48fc-a7a1-df993a1790c8 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:29.957902 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b205671e-ecee-48fc-a7a1-df993a1790c8 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:29.974306 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0aa94c-bd15-4c5c-a481-e4a9a7bb474f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:29.974306 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0aa94c-bd15-4c5c-a481-e4a9a7bb474f service neutron] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:29.974306 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0aa94c-bd15-4c5c-a481-e4a9a7bb474f service neutron] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:29.974306 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0aa94c-bd15-4c5c-a481-e4a9a7bb474f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:29.977521 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8a7908-3803-4258-9371-0ccca866f760 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:29.979378 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0aa94c-bd15-4c5c-a481-e4a9a7bb474f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:29.982765 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0aa94c-bd15-4c5c-a481-e4a9a7bb474f service neutron] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:29.985194 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec359d2-bfd0-4e49-9b2e-8275eb560a58 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4f49c1b08c460f8ba6c451c9c92c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:29.986432 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec359d2-bfd0-4e49-9b2e-8275eb560a58 service nova] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:29.987172 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec359d2-bfd0-4e49-9b2e-8275eb560a58 service nova] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:29.987643 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec359d2-bfd0-4e49-9b2e-8275eb560a58 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:29.999048 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 818/1624] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:40:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:40:30.000875 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0aa94c-bd15-4c5c-a481-e4a9a7bb474f service neutron] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:30.014187 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0aa94c-bd15-4c5c-a481-e4a9a7bb474f service neutron] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:30.021631 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19f3d95-11ef-4605-a537-3033c520c0e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:30.022416 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19f3d95-11ef-4605-a537-3033c520c0e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:30.023060 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19f3d95-11ef-4605-a537-3033c520c0e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:30.023416 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19f3d95-11ef-4605-a537-3033c520c0e9 admin admin] PATH_INFO: `/v3/projects/fa25011931e84a71bd1b20a9f3766f54/users/5919707e716e4ef28b5fe800c29167cc/roles/486144ad1bcd417e86acdbc1d14327bf` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:30.027098 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19f3d95-11ef-4605-a537-3033c520c0e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa25011931e84a71bd1b20a9f3766f54, user_id=5919707e716e4ef28b5fe800c29167cc, role_id=486144ad1bcd417e86acdbc1d14327bf)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:30.030631 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19f3d95-11ef-4605-a537-3033c520c0e9 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:30.049565 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 819/1625] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:40:29 2022] PUT /identity/v3/projects/fa25011931e84a71bd1b20a9f3766f54/users/5919707e716e4ef28b5fe800c29167cc/roles/486144ad1bcd417e86acdbc1d14327bf => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:40:30.050282 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19f3d95-11ef-4605-a537-3033c520c0e9 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:30.059677 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec359d2-bfd0-4e49-9b2e-8275eb560a58 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:30.061645 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec359d2-bfd0-4e49-9b2e-8275eb560a58 service nova] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:30.075548 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 807/1626] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:40:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:40:30.076896 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec359d2-bfd0-4e49-9b2e-8275eb560a58 service nova] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:30.089567 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec359d2-bfd0-4e49-9b2e-8275eb560a58 service nova] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:30.115640 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:40:30.195231 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3a98de-24a2-42ae-9390-b6a6a14ed703 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4f49c1b08c460f8ba6c451c9c92c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:30.195957 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3a98de-24a2-42ae-9390-b6a6a14ed703 service nova] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:30.196335 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3a98de-24a2-42ae-9390-b6a6a14ed703 service nova] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:30.196722 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3a98de-24a2-42ae-9390-b6a6a14ed703 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:30.199413 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19f3d95-11ef-4605-a537-3033c520c0e9 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:30.211215 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4292f15d-d29a-4362-8c26-ae806da6443f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:30.211847 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4292f15d-d29a-4362-8c26-ae806da6443f admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:30.212130 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4292f15d-d29a-4362-8c26-ae806da6443f admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:30.212405 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4292f15d-d29a-4362-8c26-ae806da6443f admin admin] PATH_INFO: `/v3/projects` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:30.215228 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4292f15d-d29a-4362-8c26-ae806da6443f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:30.218857 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4292f15d-d29a-4362-8c26-ae806da6443f admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:30.289620 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3a98de-24a2-42ae-9390-b6a6a14ed703 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:30.293096 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3a98de-24a2-42ae-9390-b6a6a14ed703 service nova] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:30.307100 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 808/1627] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:40:30 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:40:30.308623 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d491cd4-5617-44a7-968b-05e564641299 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:30.309035 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d491cd4-5617-44a7-968b-05e564641299 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:30.309391 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d491cd4-5617-44a7-968b-05e564641299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:30.351898 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-3d491cd4-5617-44a7-968b-05e564641299 None None] MFA Rules not processed for user `5919707e716e4ef28b5fe800c29167cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:40:30.354327 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-3d491cd4-5617-44a7-968b-05e564641299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:30.452080 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 820/1628] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:40:30 2022] POST /identity/v3/projects => generated 385 bytes in 402 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:40:30.460081 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4292f15d-d29a-4362-8c26-ae806da6443f admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:30.535323 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 809/1629] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:40:30 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:40:30.541201 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d491cd4-5617-44a7-968b-05e564641299 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:30.666012 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4292f15d-d29a-4362-8c26-ae806da6443f admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:30.682936 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65efde5-2514-4729-805a-3b03f1ac6c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:30.683729 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65efde5-2514-4729-805a-3b03f1ac6c39 admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:30.684044 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65efde5-2514-4729-805a-3b03f1ac6c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:30.684194 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d491cd4-5617-44a7-968b-05e564641299 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:30.684362 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65efde5-2514-4729-805a-3b03f1ac6c39 admin admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:30.686287 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65efde5-2514-4729-805a-3b03f1ac6c39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:30.688267 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65efde5-2514-4729-805a-3b03f1ac6c39 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:30.782503 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f02a82-eed2-4efe-b1dc-5fc53e65760b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36639dce4f97412f89d71367f650b901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:30.784017 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f02a82-eed2-4efe-b1dc-5fc53e65760b service cinder] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:30.784432 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f02a82-eed2-4efe-b1dc-5fc53e65760b service cinder] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:30.784861 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f02a82-eed2-4efe-b1dc-5fc53e65760b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:31.042497 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f02a82-eed2-4efe-b1dc-5fc53e65760b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:31.042497 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f02a82-eed2-4efe-b1dc-5fc53e65760b service cinder] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:31.043231 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 821/1630] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:40:30 2022] POST /identity/v3/users => generated 299 bytes in 484 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:40:31.043231 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c2cf40-0f30-4bbb-be1b-bfe44392880f None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:31.043231 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c2cf40-0f30-4bbb-be1b-bfe44392880f None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:31.043231 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c2cf40-0f30-4bbb-be1b-bfe44392880f None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:31.043231 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 822/1631] 10.222.0.47 () {54 vars in 912 bytes} [Wed Aug 24 06:40:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:40:31.043231 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05bd737-60af-453a-a402-a7f81b5c578d None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:31.043231 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05bd737-60af-453a-a402-a7f81b5c578d None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:31.043948 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05bd737-60af-453a-a402-a7f81b5c578d None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:31.043948 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 823/1632] 10.222.0.47 () {54 vars in 912 bytes} [Wed Aug 24 06:40:30 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:40:31.043948 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f906ef76-721e-4b13-b552-b0ce4f74fc14 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:31.043948 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f906ef76-721e-4b13-b552-b0ce4f74fc14 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:31.043948 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f906ef76-721e-4b13-b552-b0ce4f74fc14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:31.044735 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 810/1633] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:40:30 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:40:31.044735 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f02a82-eed2-4efe-b1dc-5fc53e65760b service cinder] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:31.044735 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f02a82-eed2-4efe-b1dc-5fc53e65760b service cinder] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:31.045057 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-f906ef76-721e-4b13-b552-b0ce4f74fc14 None None] MFA Rules not processed for user `1f4f49c1b08c460f8ba6c451c9c92c2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:40:31.045136 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcde963c-c9e6-45bb-9a76-210ef20465e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:31.045260 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-f906ef76-721e-4b13-b552-b0ce4f74fc14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:31.045348 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde963c-c9e6-45bb-9a76-210ef20465e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:31.045348 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde963c-c9e6-45bb-9a76-210ef20465e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:31.045348 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde963c-c9e6-45bb-9a76-210ef20465e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:31.045348 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcde963c-c9e6-45bb-9a76-210ef20465e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:31.045348 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcde963c-c9e6-45bb-9a76-210ef20465e8 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:31.045348 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 811/1634] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:40:30 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:40:31.045764 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8843300-3075-483b-ba99-ae5e861d7330 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:31.045764 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8843300-3075-483b-ba99-ae5e861d7330 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:31.045764 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8843300-3075-483b-ba99-ae5e861d7330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:31.086659 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-d8843300-3075-483b-ba99-ae5e861d7330 None None] MFA Rules not processed for user `1f4f49c1b08c460f8ba6c451c9c92c2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:40:31.089824 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-d8843300-3075-483b-ba99-ae5e861d7330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:31.115408 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:40:31.141344 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 824/1635] 10.222.0.47 () {58 vars in 1009 bytes} [Wed Aug 24 06:40:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:40:31.143257 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f906ef76-721e-4b13-b552-b0ce4f74fc14 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:31.156841 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f906ef76-721e-4b13-b552-b0ce4f74fc14 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:31.163836 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6015b4-f422-4cf6-98ae-6f3ddbe94175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:31.165276 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6015b4-f422-4cf6-98ae-6f3ddbe94175 admin admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:31.165561 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6015b4-f422-4cf6-98ae-6f3ddbe94175 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:31.165835 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6015b4-f422-4cf6-98ae-6f3ddbe94175 admin admin] PATH_INFO: `/v3/projects/d0ec2d305f40451d804cccca8ae3ce57/users/39bd73bf2048402ea35d6acb46670e31/roles/486144ad1bcd417e86acdbc1d14327bf` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:31.186581 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6015b4-f422-4cf6-98ae-6f3ddbe94175 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0ec2d305f40451d804cccca8ae3ce57, user_id=39bd73bf2048402ea35d6acb46670e31, role_id=486144ad1bcd417e86acdbc1d14327bf)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:31.189902 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6015b4-f422-4cf6-98ae-6f3ddbe94175 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:31.207619 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 812/1636] 10.222.0.47 () {58 vars in 1009 bytes} [Wed Aug 24 06:40:31 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:40:31.208673 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8843300-3075-483b-ba99-ae5e861d7330 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:31.219816 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8843300-3075-483b-ba99-ae5e861d7330 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:31.227764 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f704b8-d2bf-4ab4-989d-4ad82071cd84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36639dce4f97412f89d71367f650b901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:31.228475 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f704b8-d2bf-4ab4-989d-4ad82071cd84 service cinder] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:31.229057 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f704b8-d2bf-4ab4-989d-4ad82071cd84 service cinder] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:31.229706 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f704b8-d2bf-4ab4-989d-4ad82071cd84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:31.241460 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f704b8-d2bf-4ab4-989d-4ad82071cd84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:31.243025 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f704b8-d2bf-4ab4-989d-4ad82071cd84 service cinder] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:31.256338 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 813/1637] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:40:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:40:31.352370 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f704b8-d2bf-4ab4-989d-4ad82071cd84 service cinder] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:31.369767 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f704b8-d2bf-4ab4-989d-4ad82071cd84 service cinder] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:31.374524 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08383ee7-8189-4d4c-85f5-379a05e1c6c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36639dce4f97412f89d71367f650b901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:31.375261 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08383ee7-8189-4d4c-85f5-379a05e1c6c3 service cinder] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:31.375608 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08383ee7-8189-4d4c-85f5-379a05e1c6c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:31.375961 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08383ee7-8189-4d4c-85f5-379a05e1c6c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:31.391713 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08383ee7-8189-4d4c-85f5-379a05e1c6c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:31.391713 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08383ee7-8189-4d4c-85f5-379a05e1c6c3 service cinder] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:31.405476 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 825/1638] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:40:31 2022] PUT /identity/v3/projects/d0ec2d305f40451d804cccca8ae3ce57/users/39bd73bf2048402ea35d6acb46670e31/roles/486144ad1bcd417e86acdbc1d14327bf => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:40:31.406844 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6015b4-f422-4cf6-98ae-6f3ddbe94175 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:31.417457 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 814/1639] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:40:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:40:31.491802 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6015b4-f422-4cf6-98ae-6f3ddbe94175 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:31.496702 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9c343e-2e62-448a-ae50-1b924545fe1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:31.497809 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9c343e-2e62-448a-ae50-1b924545fe1c admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:31.498406 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9c343e-2e62-448a-ae50-1b924545fe1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:31.499122 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9c343e-2e62-448a-ae50-1b924545fe1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:31.500043 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9c343e-2e62-448a-ae50-1b924545fe1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:31.503062 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9c343e-2e62-448a-ae50-1b924545fe1c admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:31.527854 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 826/1640] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:40:31 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:40:31.528168 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08383ee7-8189-4d4c-85f5-379a05e1c6c3 service cinder] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:31.547689 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08383ee7-8189-4d4c-85f5-379a05e1c6c3 service cinder] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:31.560879 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26cb160-ddc9-4081-bd4b-061d1304089e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:31.561662 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26cb160-ddc9-4081-bd4b-061d1304089e admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:31.561977 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26cb160-ddc9-4081-bd4b-061d1304089e admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:31.562255 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26cb160-ddc9-4081-bd4b-061d1304089e admin admin] PATH_INFO: `/v3/projects/d0ec2d305f40451d804cccca8ae3ce57/users/39bd73bf2048402ea35d6acb46670e31/roles/486144ad1bcd417e86acdbc1d14327bf` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:31.568992 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26cb160-ddc9-4081-bd4b-061d1304089e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0ec2d305f40451d804cccca8ae3ce57, user_id=39bd73bf2048402ea35d6acb46670e31, role_id=486144ad1bcd417e86acdbc1d14327bf)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:31.571938 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26cb160-ddc9-4081-bd4b-061d1304089e admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:31.601230 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 815/1641] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:40:31 2022] PUT /identity/v3/projects/d0ec2d305f40451d804cccca8ae3ce57/users/39bd73bf2048402ea35d6acb46670e31/roles/486144ad1bcd417e86acdbc1d14327bf => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:40:32.134192 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b92682-5dab-4f7f-a788-d1b8b51db188 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:32.134192 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b92682-5dab-4f7f-a788-d1b8b51db188 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:32.147478 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9c343e-2e62-448a-ae50-1b924545fe1c admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:32.155769 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b92682-5dab-4f7f-a788-d1b8b51db188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:32.155769 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-91b92682-5dab-4f7f-a788-d1b8b51db188 None None] MFA Rules not processed for user `39bd73bf2048402ea35d6acb46670e31`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:40:32.155769 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-91b92682-5dab-4f7f-a788-d1b8b51db188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:32.164939 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9c343e-2e62-448a-ae50-1b924545fe1c admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:32.167758 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 816/1642] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:40:31 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:40:32.169015 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214ebf26-316d-498b-91da-4e7c3392aee9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36639dce4f97412f89d71367f650b901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:32.169015 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ebf26-316d-498b-91da-4e7c3392aee9 service cinder] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:32.169015 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ebf26-316d-498b-91da-4e7c3392aee9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:32.169015 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ebf26-316d-498b-91da-4e7c3392aee9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:32.171214 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214ebf26-316d-498b-91da-4e7c3392aee9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:32.171214 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214ebf26-316d-498b-91da-4e7c3392aee9 service cinder] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:32.171214 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 827/1643] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:40:31 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:40:33.145144 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:40:33.294043 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b92682-5dab-4f7f-a788-d1b8b51db188 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:33.308044 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b92682-5dab-4f7f-a788-d1b8b51db188 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:33.320662 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b607b169-5f89-451b-9c7f-668588ec3ac2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36639dce4f97412f89d71367f650b901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:33.321461 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b607b169-5f89-451b-9c7f-668588ec3ac2 service cinder] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:33.321874 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b607b169-5f89-451b-9c7f-668588ec3ac2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:33.322279 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b607b169-5f89-451b-9c7f-668588ec3ac2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:33.333144 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214ebf26-316d-498b-91da-4e7c3392aee9 service cinder] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:33.354050 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b607b169-5f89-451b-9c7f-668588ec3ac2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:33.356230 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b607b169-5f89-451b-9c7f-668588ec3ac2 service cinder] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:33.388985 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 817/1644] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:40:33 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:40:33.471175 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214ebf26-316d-498b-91da-4e7c3392aee9 service cinder] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:33.482391 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8c4352-e9f3-4796-96b7-335fa57dfac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:33.483543 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8c4352-e9f3-4796-96b7-335fa57dfac8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:33.483961 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8c4352-e9f3-4796-96b7-335fa57dfac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:33.484516 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8c4352-e9f3-4796-96b7-335fa57dfac8 admin admin] PATH_INFO: `/v3/users/a4fddb5f283940838ab1e60ec8c64a9c` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:33.487884 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8c4352-e9f3-4796-96b7-335fa57dfac8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4fddb5f283940838ab1e60ec8c64a9c)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:33.489627 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: WARNING py.warnings [None req-bd8c4352-e9f3-4796-96b7-335fa57dfac8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 06:40:33.489627 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: warnings.warn(msg) Aug 24 06:40:33.489627 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: Aug 24 06:40:33.491383 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8c4352-e9f3-4796-96b7-335fa57dfac8 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:33.615224 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:40:33.747561 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b607b169-5f89-451b-9c7f-668588ec3ac2 service cinder] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:33.761220 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b607b169-5f89-451b-9c7f-668588ec3ac2 service cinder] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:33.767247 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7504ee-77f9-4de3-8e1b-ec91359f8f4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:33.767804 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7504ee-77f9-4de3-8e1b-ec91359f8f4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:33.768247 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7504ee-77f9-4de3-8e1b-ec91359f8f4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:33.768844 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7504ee-77f9-4de3-8e1b-ec91359f8f4a admin admin] PATH_INFO: `/v3/users/d787466a655045c08ebe94eb920088a9` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:33.770579 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7504ee-77f9-4de3-8e1b-ec91359f8f4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d787466a655045c08ebe94eb920088a9)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:33.772838 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7504ee-77f9-4de3-8e1b-ec91359f8f4a admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:34.091802 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-bd8c4352-e9f3-4796-96b7-335fa57dfac8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4fddb5f283940838ab1e60ec8c64a9c'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:40:34.092901 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-bd8c4352-e9f3-4796-96b7-335fa57dfac8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4fddb5f283940838ab1e60ec8c64a9c'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:40:34.095401 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-bd8c4352-e9f3-4796-96b7-335fa57dfac8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4fddb5f283940838ab1e60ec8c64a9c'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:40:34.208981 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-2e7504ee-77f9-4de3-8e1b-ec91359f8f4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd787466a655045c08ebe94eb920088a9'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:40:34.209903 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-2e7504ee-77f9-4de3-8e1b-ec91359f8f4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd787466a655045c08ebe94eb920088a9'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:40:34.210550 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-2e7504ee-77f9-4de3-8e1b-ec91359f8f4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd787466a655045c08ebe94eb920088a9'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:40:34.218844 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-2e7504ee-77f9-4de3-8e1b-ec91359f8f4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd787466a655045c08ebe94eb920088a9'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:40:34.302794 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-bd8c4352-e9f3-4796-96b7-335fa57dfac8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4fddb5f283940838ab1e60ec8c64a9c'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:40:34.333369 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-bd8c4352-e9f3-4796-96b7-335fa57dfac8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4fddb5f283940838ab1e60ec8c64a9c'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:40:34.358551 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 828/1645] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:40:33 2022] DELETE /identity/v3/users/a4fddb5f283940838ab1e60ec8c64a9c => generated 0 bytes in 1022 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:40:34.360470 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe4f6a1-61d9-4fdd-a62a-80be3587e1c5 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:34.361401 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe4f6a1-61d9-4fdd-a62a-80be3587e1c5 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:34.362016 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe4f6a1-61d9-4fdd-a62a-80be3587e1c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:34.386808 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-1fe4f6a1-61d9-4fdd-a62a-80be3587e1c5 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:40:34.388220 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-1fe4f6a1-61d9-4fdd-a62a-80be3587e1c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:34.477418 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 829/1646] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:40:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:40:34.490435 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe4f6a1-61d9-4fdd-a62a-80be3587e1c5 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:34.491980 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-1fe4f6a1-61d9-4fdd-a62a-80be3587e1c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:34.504969 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-2e7504ee-77f9-4de3-8e1b-ec91359f8f4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd787466a655045c08ebe94eb920088a9'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:40:34.540165 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 818/1647] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:40:33 2022] DELETE /identity/v3/users/d787466a655045c08ebe94eb920088a9 => generated 0 bytes in 793 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:40:34.546148 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280ca712-f45a-42fc-af86-38cc7cd380c4 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:34.546437 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280ca712-f45a-42fc-af86-38cc7cd380c4 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:34.546714 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280ca712-f45a-42fc-af86-38cc7cd380c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:34.577673 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe4f6a1-61d9-4fdd-a62a-80be3587e1c5 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:34.591552 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-280ca712-f45a-42fc-af86-38cc7cd380c4 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:40:34.594331 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-280ca712-f45a-42fc-af86-38cc7cd380c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:34.710062 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba67030-6580-4fbb-8357-4ffd74323c4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:34.710062 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba67030-6580-4fbb-8357-4ffd74323c4b service neutron] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:34.710062 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba67030-6580-4fbb-8357-4ffd74323c4b service neutron] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:34.710062 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba67030-6580-4fbb-8357-4ffd74323c4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:34.745333 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 819/1648] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:40:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:40:34.791310 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280ca712-f45a-42fc-af86-38cc7cd380c4 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:34.792702 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-280ca712-f45a-42fc-af86-38cc7cd380c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:34.835680 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280ca712-f45a-42fc-af86-38cc7cd380c4 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:34.842428 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1dc12e-161d-4792-b203-3b66757e234f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:34.845134 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1dc12e-161d-4792-b203-3b66757e234f service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:34.845981 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1dc12e-161d-4792-b203-3b66757e234f service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:34.846434 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1dc12e-161d-4792-b203-3b66757e234f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:34.867185 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba67030-6580-4fbb-8357-4ffd74323c4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:34.868933 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba67030-6580-4fbb-8357-4ffd74323c4b service neutron] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:34.890395 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 830/1649] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:40:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:40:34.915551 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:40:34.925814 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1dc12e-161d-4792-b203-3b66757e234f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:34.933773 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1dc12e-161d-4792-b203-3b66757e234f service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:34.962943 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 820/1650] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:40:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:40:35.892343 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:40:36.341708 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba67030-6580-4fbb-8357-4ffd74323c4b service neutron] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:36.344311 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-7ba67030-6580-4fbb-8357-4ffd74323c4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:36.379761 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba67030-6580-4fbb-8357-4ffd74323c4b service neutron] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:36.405963 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc038765-97e6-4269-a12a-aed3fdf3f5a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:36.405963 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc038765-97e6-4269-a12a-aed3fdf3f5a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:36.405963 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc038765-97e6-4269-a12a-aed3fdf3f5a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:36.405963 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc038765-97e6-4269-a12a-aed3fdf3f5a4 admin admin] PATH_INFO: `/v3/projects/738113e0ae1c4cfa942acc615e748a9b` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:36.408129 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc038765-97e6-4269-a12a-aed3fdf3f5a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=738113e0ae1c4cfa942acc615e748a9b)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:36.410750 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc038765-97e6-4269-a12a-aed3fdf3f5a4 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:36.433518 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:40:36.686575 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-cc038765-97e6-4269-a12a-aed3fdf3f5a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '738113e0ae1c4cfa942acc615e748a9b', 'request_id': 'req-cc038765-97e6-4269-a12a-aed3fdf3f5a4'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:40:36.817307 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-cc038765-97e6-4269-a12a-aed3fdf3f5a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '738113e0ae1c4cfa942acc615e748a9b', 'request_id': 'req-cc038765-97e6-4269-a12a-aed3fdf3f5a4'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:40:36.826759 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-cc038765-97e6-4269-a12a-aed3fdf3f5a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '738113e0ae1c4cfa942acc615e748a9b', 'request_id': 'req-cc038765-97e6-4269-a12a-aed3fdf3f5a4'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:40:36.856949 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-cc038765-97e6-4269-a12a-aed3fdf3f5a4 admin admin] The token cache is being invalidate because project 738113e0ae1c4cfa942acc615e748a9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81723) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:40:36.862627 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-cc038765-97e6-4269-a12a-aed3fdf3f5a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:40:36.865028 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-cc038765-97e6-4269-a12a-aed3fdf3f5a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:40:36.886684 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 831/1651] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:40:36 2022] DELETE /identity/v3/projects/738113e0ae1c4cfa942acc615e748a9b => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:40:36.886776 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628e539c-efcc-4e7d-8621-da74687bdc45 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:36.886776 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628e539c-efcc-4e7d-8621-da74687bdc45 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:36.886776 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628e539c-efcc-4e7d-8621-da74687bdc45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:36.899098 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-628e539c-efcc-4e7d-8621-da74687bdc45 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:40:36.900718 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-628e539c-efcc-4e7d-8621-da74687bdc45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:37.048553 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 821/1652] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:40:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:40:37.059459 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88096f82-224b-444e-9b40-af89ff6ce283 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:37.067418 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88096f82-224b-444e-9b40-af89ff6ce283 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:37.067418 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88096f82-224b-444e-9b40-af89ff6ce283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:37.077174 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628e539c-efcc-4e7d-8621-da74687bdc45 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:37.079551 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-628e539c-efcc-4e7d-8621-da74687bdc45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:37.103998 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628e539c-efcc-4e7d-8621-da74687bdc45 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:37.122032 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdcb443-53ad-4f1a-a1be-7c13ce49e2be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:37.122700 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdcb443-53ad-4f1a-a1be-7c13ce49e2be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:37.123191 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdcb443-53ad-4f1a-a1be-7c13ce49e2be admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:37.123530 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdcb443-53ad-4f1a-a1be-7c13ce49e2be admin admin] PATH_INFO: `/v3/projects/2d2ff0314b9846f9af2f6a72f8d0e5d5` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:37.126252 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdcb443-53ad-4f1a-a1be-7c13ce49e2be admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d2ff0314b9846f9af2f6a72f8d0e5d5)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:37.128935 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: WARNING py.warnings [None req-3cdcb443-53ad-4f1a-a1be-7c13ce49e2be admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 06:40:37.128935 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: warnings.warn(msg) Aug 24 06:40:37.128935 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: Aug 24 06:40:37.130219 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdcb443-53ad-4f1a-a1be-7c13ce49e2be admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:37.141139 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-88096f82-224b-444e-9b40-af89ff6ce283 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:40:37.146809 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-88096f82-224b-444e-9b40-af89ff6ce283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:37.207261 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 832/1653] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:40:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:40:37.223733 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88096f82-224b-444e-9b40-af89ff6ce283 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:37.245078 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88096f82-224b-444e-9b40-af89ff6ce283 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:37.264846 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e4a957-764d-4255-a51d-a35456cbabd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:37.264846 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e4a957-764d-4255-a51d-a35456cbabd6 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:37.264846 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e4a957-764d-4255-a51d-a35456cbabd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:37.264846 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e4a957-764d-4255-a51d-a35456cbabd6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:37.269634 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e4a957-764d-4255-a51d-a35456cbabd6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:40:37.270284 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e4a957-764d-4255-a51d-a35456cbabd6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:37.272316 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e4a957-764d-4255-a51d-a35456cbabd6 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:37.307471 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 833/1654] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 24 06:40:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:40:37.312680 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e4a957-764d-4255-a51d-a35456cbabd6 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:37.334027 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e4a957-764d-4255-a51d-a35456cbabd6 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:37.351124 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7b5a43-16bc-4bdc-a8ec-5670b22730c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:37.352076 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7b5a43-16bc-4bdc-a8ec-5670b22730c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:37.352901 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7b5a43-16bc-4bdc-a8ec-5670b22730c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:37.353373 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7b5a43-16bc-4bdc-a8ec-5670b22730c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:37.354457 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7b5a43-16bc-4bdc-a8ec-5670b22730c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:37.358792 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7b5a43-16bc-4bdc-a8ec-5670b22730c1 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:37.477066 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-3cdcb443-53ad-4f1a-a1be-7c13ce49e2be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d2ff0314b9846f9af2f6a72f8d0e5d5', 'request_id': 'req-3cdcb443-53ad-4f1a-a1be-7c13ce49e2be'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:40:37.490466 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-3cdcb443-53ad-4f1a-a1be-7c13ce49e2be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d2ff0314b9846f9af2f6a72f8d0e5d5', 'request_id': 'req-3cdcb443-53ad-4f1a-a1be-7c13ce49e2be'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:40:37.554377 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 834/1655] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:40:37 2022] POST /identity/v3/projects => generated 405 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:40:37.558213 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7b5a43-16bc-4bdc-a8ec-5670b22730c1 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:37.621494 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-3cdcb443-53ad-4f1a-a1be-7c13ce49e2be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d2ff0314b9846f9af2f6a72f8d0e5d5', 'request_id': 'req-3cdcb443-53ad-4f1a-a1be-7c13ce49e2be'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:40:37.725342 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-3cdcb443-53ad-4f1a-a1be-7c13ce49e2be admin admin] The token cache is being invalidate because project 2d2ff0314b9846f9af2f6a72f8d0e5d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:40:37.729447 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-3cdcb443-53ad-4f1a-a1be-7c13ce49e2be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:40:37.730925 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-3cdcb443-53ad-4f1a-a1be-7c13ce49e2be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:40:37.734011 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 822/1656] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:40:37 2022] DELETE /identity/v3/projects/2d2ff0314b9846f9af2f6a72f8d0e5d5 => generated 0 bytes in 657 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:40:37.735768 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdcb443-53ad-4f1a-a1be-7c13ce49e2be admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:37.737938 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-3cdcb443-53ad-4f1a-a1be-7c13ce49e2be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:37.739716 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7b5a43-16bc-4bdc-a8ec-5670b22730c1 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:37.763356 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed6c3a5-0268-4d21-bbff-d1e844e163dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:37.764067 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed6c3a5-0268-4d21-bbff-d1e844e163dd admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:37.764358 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed6c3a5-0268-4d21-bbff-d1e844e163dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:37.764635 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed6c3a5-0268-4d21-bbff-d1e844e163dd admin admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:37.765310 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed6c3a5-0268-4d21-bbff-d1e844e163dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:37.767854 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed6c3a5-0268-4d21-bbff-d1e844e163dd admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:37.978448 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdcb443-53ad-4f1a-a1be-7c13ce49e2be admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:38.009916 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866de9a7-2c2b-4765-bb9b-de2a8f8d2339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:38.009916 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866de9a7-2c2b-4765-bb9b-de2a8f8d2339 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:38.009916 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866de9a7-2c2b-4765-bb9b-de2a8f8d2339 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:38.009916 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866de9a7-2c2b-4765-bb9b-de2a8f8d2339 admin admin] PATH_INFO: `/v3/users/4125b9dd74a842bca7230e28171b226b` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:38.010478 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866de9a7-2c2b-4765-bb9b-de2a8f8d2339 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4125b9dd74a842bca7230e28171b226b)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:38.010478 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866de9a7-2c2b-4765-bb9b-de2a8f8d2339 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:38.143703 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 835/1657] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:40:37 2022] POST /identity/v3/users => generated 303 bytes in 580 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:40:38.143703 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed6c3a5-0268-4d21-bbff-d1e844e163dd admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:38.150191 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-2ed6c3a5-0268-4d21-bbff-d1e844e163dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:38.171592 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed6c3a5-0268-4d21-bbff-d1e844e163dd admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:38.183131 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a9a788-0870-4c29-a67d-c9996bdf3b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:38.184087 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a9a788-0870-4c29-a67d-c9996bdf3b89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:38.184551 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a9a788-0870-4c29-a67d-c9996bdf3b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:38.185068 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a9a788-0870-4c29-a67d-c9996bdf3b89 admin admin] PATH_INFO: `/v3/users/39bd73bf2048402ea35d6acb46670e31` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:38.189212 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:40:38.196286 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a9a788-0870-4c29-a67d-c9996bdf3b89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=39bd73bf2048402ea35d6acb46670e31)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:38.223886 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a9a788-0870-4c29-a67d-c9996bdf3b89 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:38.395298 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-866de9a7-2c2b-4765-bb9b-de2a8f8d2339 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4125b9dd74a842bca7230e28171b226b'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:40:38.395981 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-866de9a7-2c2b-4765-bb9b-de2a8f8d2339 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4125b9dd74a842bca7230e28171b226b'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:40:38.396622 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-866de9a7-2c2b-4765-bb9b-de2a8f8d2339 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4125b9dd74a842bca7230e28171b226b'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:40:38.422761 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-866de9a7-2c2b-4765-bb9b-de2a8f8d2339 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4125b9dd74a842bca7230e28171b226b'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:40:38.595919 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-866de9a7-2c2b-4765-bb9b-de2a8f8d2339 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4125b9dd74a842bca7230e28171b226b'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:40:38.632300 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-67a9a788-0870-4c29-a67d-c9996bdf3b89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39bd73bf2048402ea35d6acb46670e31'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:40:38.633516 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-67a9a788-0870-4c29-a67d-c9996bdf3b89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39bd73bf2048402ea35d6acb46670e31'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:40:38.634337 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-67a9a788-0870-4c29-a67d-c9996bdf3b89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39bd73bf2048402ea35d6acb46670e31'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:40:38.672187 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 823/1658] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:40:37 2022] DELETE /identity/v3/users/4125b9dd74a842bca7230e28171b226b => generated 0 bytes in 916 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:40:38.672187 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866de9a7-2c2b-4765-bb9b-de2a8f8d2339 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:38.672187 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-866de9a7-2c2b-4765-bb9b-de2a8f8d2339 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:38.805334 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866de9a7-2c2b-4765-bb9b-de2a8f8d2339 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:38.810747 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb0ba5a-896b-44a4-a4bf-72525e7f4f6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:38.811386 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb0ba5a-896b-44a4-a4bf-72525e7f4f6a admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:38.811671 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb0ba5a-896b-44a4-a4bf-72525e7f4f6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:38.812014 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb0ba5a-896b-44a4-a4bf-72525e7f4f6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:38.812786 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb0ba5a-896b-44a4-a4bf-72525e7f4f6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:38.815166 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb0ba5a-896b-44a4-a4bf-72525e7f4f6a admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:38.838275 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 824/1659] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:40:38 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:40:38.840105 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06a6818-3221-4f2f-b518-0cca47336839 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:38.840621 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06a6818-3221-4f2f-b518-0cca47336839 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:38.841089 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06a6818-3221-4f2f-b518-0cca47336839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:38.844906 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:40:38.846006 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-67a9a788-0870-4c29-a67d-c9996bdf3b89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39bd73bf2048402ea35d6acb46670e31'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:40:38.887189 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-67a9a788-0870-4c29-a67d-c9996bdf3b89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39bd73bf2048402ea35d6acb46670e31'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:40:38.898440 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-d06a6818-3221-4f2f-b518-0cca47336839 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:40:38.903440 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-d06a6818-3221-4f2f-b518-0cca47336839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:38.917527 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 836/1660] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:40:38 2022] DELETE /identity/v3/users/39bd73bf2048402ea35d6acb46670e31 => generated 0 bytes in 773 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:40:38.919045 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a9a788-0870-4c29-a67d-c9996bdf3b89 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:38.921813 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-67a9a788-0870-4c29-a67d-c9996bdf3b89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:38.926827 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 825/1661] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:40:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:40:38.969255 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06a6818-3221-4f2f-b518-0cca47336839 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:38.972187 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-d06a6818-3221-4f2f-b518-0cca47336839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:39.103573 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a9a788-0870-4c29-a67d-c9996bdf3b89 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:39.113262 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6312230-a207-4f03-9342-433f524c8333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:39.114075 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6312230-a207-4f03-9342-433f524c8333 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:39.114896 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6312230-a207-4f03-9342-433f524c8333 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:39.115193 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6312230-a207-4f03-9342-433f524c8333 admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:39.116211 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6312230-a207-4f03-9342-433f524c8333 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:39.118330 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6312230-a207-4f03-9342-433f524c8333 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:39.128269 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06a6818-3221-4f2f-b518-0cca47336839 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:39.139677 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 837/1662] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:40:38 2022] GET /identity/v3/roles => generated 1420 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:40:39.140945 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6312230-a207-4f03-9342-433f524c8333 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:39.157777 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6312230-a207-4f03-9342-433f524c8333 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:39.169932 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53639a0-6f36-4ec0-afc6-c557272dec42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:39.173455 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53639a0-6f36-4ec0-afc6-c557272dec42 admin admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:39.173455 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53639a0-6f36-4ec0-afc6-c557272dec42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:39.173455 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53639a0-6f36-4ec0-afc6-c557272dec42 admin admin] PATH_INFO: `/v3/projects/0e2df09f149a4902be90ebd39ff1988a/users/b3e9b8a8ac12424f8cbf2e4fba9b4e28/roles/e1f9b167c51c4a388cbc9594db8a465d` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:39.196582 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53639a0-6f36-4ec0-afc6-c557272dec42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e2df09f149a4902be90ebd39ff1988a, user_id=b3e9b8a8ac12424f8cbf2e4fba9b4e28, role_id=e1f9b167c51c4a388cbc9594db8a465d)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:39.199377 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53639a0-6f36-4ec0-afc6-c557272dec42 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:39.264536 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbb6bbd-e732-4ce0-82c9-cae5b5258991 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:39.265498 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb6bbd-e732-4ce0-82c9-cae5b5258991 service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:39.265987 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb6bbd-e732-4ce0-82c9-cae5b5258991 service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:39.267559 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbb6bbd-e732-4ce0-82c9-cae5b5258991 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:39.314575 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbb6bbd-e732-4ce0-82c9-cae5b5258991 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:39.316581 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbb6bbd-e732-4ce0-82c9-cae5b5258991 service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:39.333942 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 826/1663] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:40:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:40:39.372974 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 838/1664] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:40:39 2022] PUT /identity/v3/projects/0e2df09f149a4902be90ebd39ff1988a/users/b3e9b8a8ac12424f8cbf2e4fba9b4e28/roles/e1f9b167c51c4a388cbc9594db8a465d => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:40:39.376691 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d431d1-be41-44af-9294-f21526133b57 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:39.377250 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d431d1-be41-44af-9294-f21526133b57 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:39.380727 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d431d1-be41-44af-9294-f21526133b57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:39.472496 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-a7d431d1-be41-44af-9294-f21526133b57 None None] MFA Rules not processed for user `b3e9b8a8ac12424f8cbf2e4fba9b4e28`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:40:39.475715 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-a7d431d1-be41-44af-9294-f21526133b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:39.662390 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 827/1665] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:40:39 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:40:39.663014 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53639a0-6f36-4ec0-afc6-c557272dec42 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:39.779538 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53639a0-6f36-4ec0-afc6-c557272dec42 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:39.787140 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15573d07-0b8b-44cb-850c-b8818c1789f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:39.788126 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15573d07-0b8b-44cb-850c-b8818c1789f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:39.788572 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15573d07-0b8b-44cb-850c-b8818c1789f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:39.788994 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15573d07-0b8b-44cb-850c-b8818c1789f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:39.789843 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15573d07-0b8b-44cb-850c-b8818c1789f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:39.792818 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15573d07-0b8b-44cb-850c-b8818c1789f4 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:39.998956 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 839/1666] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:40:39 2022] POST /identity/v3/projects => generated 405 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:40:40.000259 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d431d1-be41-44af-9294-f21526133b57 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:40.019204 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d431d1-be41-44af-9294-f21526133b57 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:40.196130 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21d6eb0-3f23-4d44-9097-2cee48ee776e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:40.197264 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21d6eb0-3f23-4d44-9097-2cee48ee776e admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:40.197774 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21d6eb0-3f23-4d44-9097-2cee48ee776e admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:40.198268 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21d6eb0-3f23-4d44-9097-2cee48ee776e admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:40.199436 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21d6eb0-3f23-4d44-9097-2cee48ee776e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:40.203347 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21d6eb0-3f23-4d44-9097-2cee48ee776e admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:40.406159 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 828/1667] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:40:39 2022] POST /identity/v3/users => generated 309 bytes in 402 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:40:40.408419 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15573d07-0b8b-44cb-850c-b8818c1789f4 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:40.428508 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15573d07-0b8b-44cb-850c-b8818c1789f4 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:40.444966 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f90b247-a89e-4b72-8927-c44f2aae1a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:40.445762 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f90b247-a89e-4b72-8927-c44f2aae1a91 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:40.446127 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f90b247-a89e-4b72-8927-c44f2aae1a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:40.446481 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f90b247-a89e-4b72-8927-c44f2aae1a91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:40.447253 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f90b247-a89e-4b72-8927-c44f2aae1a91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:40.451124 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f90b247-a89e-4b72-8927-c44f2aae1a91 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:40.485164 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21d6eb0-3f23-4d44-9097-2cee48ee776e admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:40.496214 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 840/1668] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:40:40 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:40:40.499230 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21d6eb0-3f23-4d44-9097-2cee48ee776e admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:40.510418 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:40:40.511727 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97278047-473f-47db-ac50-0ef2ba630973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:40.513214 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97278047-473f-47db-ac50-0ef2ba630973 admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:40.514244 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97278047-473f-47db-ac50-0ef2ba630973 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:40.514665 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97278047-473f-47db-ac50-0ef2ba630973 admin admin] PATH_INFO: `/v3/projects/718912837e2e43a984ca5b5437ff4ce8/users/005e9efcb6ef4102ad5493a5c666554c/roles/486144ad1bcd417e86acdbc1d14327bf` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:40.542935 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97278047-473f-47db-ac50-0ef2ba630973 admin admin] RBAC: Authorizing `identity:create_grant(project_id=718912837e2e43a984ca5b5437ff4ce8, user_id=005e9efcb6ef4102ad5493a5c666554c, role_id=486144ad1bcd417e86acdbc1d14327bf)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:40.546510 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97278047-473f-47db-ac50-0ef2ba630973 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:40.670516 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 829/1669] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:40:40 2022] PUT /identity/v3/projects/718912837e2e43a984ca5b5437ff4ce8/users/005e9efcb6ef4102ad5493a5c666554c/roles/486144ad1bcd417e86acdbc1d14327bf => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:40:40.675181 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f90b247-a89e-4b72-8927-c44f2aae1a91 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:40.690578 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f90b247-a89e-4b72-8927-c44f2aae1a91 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:40.957910 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c1eb0a-5011-4f86-8443-39721b9f9d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:40.958975 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c1eb0a-5011-4f86-8443-39721b9f9d44 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:40.959461 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c1eb0a-5011-4f86-8443-39721b9f9d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:40.961891 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c1eb0a-5011-4f86-8443-39721b9f9d44 admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:40.963414 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c1eb0a-5011-4f86-8443-39721b9f9d44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:40.974009 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c1eb0a-5011-4f86-8443-39721b9f9d44 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:41.021416 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 841/1670] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:40:40 2022] GET /identity/v3/roles => generated 1420 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:40:41.023474 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97278047-473f-47db-ac50-0ef2ba630973 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:41.041242 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97278047-473f-47db-ac50-0ef2ba630973 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:41.050787 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e712c2-86e2-4e11-81fe-2994c3f6f257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:41.051920 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e712c2-86e2-4e11-81fe-2994c3f6f257 admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:41.052444 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e712c2-86e2-4e11-81fe-2994c3f6f257 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:41.053272 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e712c2-86e2-4e11-81fe-2994c3f6f257 admin admin] PATH_INFO: `/v3/projects/718912837e2e43a984ca5b5437ff4ce8/users/005e9efcb6ef4102ad5493a5c666554c/roles/486144ad1bcd417e86acdbc1d14327bf` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:41.057388 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e712c2-86e2-4e11-81fe-2994c3f6f257 admin admin] RBAC: Authorizing `identity:create_grant(project_id=718912837e2e43a984ca5b5437ff4ce8, user_id=005e9efcb6ef4102ad5493a5c666554c, role_id=486144ad1bcd417e86acdbc1d14327bf)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:41.065289 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e712c2-86e2-4e11-81fe-2994c3f6f257 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:41.082163 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 830/1671] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:40:41 2022] PUT /identity/v3/projects/718912837e2e43a984ca5b5437ff4ce8/users/005e9efcb6ef4102ad5493a5c666554c/roles/486144ad1bcd417e86acdbc1d14327bf => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:40:41.103092 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3a1609-111a-4538-b685-64f24a80d666 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:41.103408 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3a1609-111a-4538-b685-64f24a80d666 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:41.103663 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3a1609-111a-4538-b685-64f24a80d666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:41.160450 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-7d3a1609-111a-4538-b685-64f24a80d666 None None] MFA Rules not processed for user `005e9efcb6ef4102ad5493a5c666554c`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:40:41.162449 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-7d3a1609-111a-4538-b685-64f24a80d666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:41.279962 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 842/1672] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:40:41 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:40:41.292706 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e712c2-86e2-4e11-81fe-2994c3f6f257 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:41.295202 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-a9e712c2-86e2-4e11-81fe-2994c3f6f257 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:41.560554 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e712c2-86e2-4e11-81fe-2994c3f6f257 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:41.570491 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d132d46d-ecc4-4209-8e9d-dd77217ebb76 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36639dce4f97412f89d71367f650b901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:41.571137 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d132d46d-ecc4-4209-8e9d-dd77217ebb76 service cinder] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:41.571511 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d132d46d-ecc4-4209-8e9d-dd77217ebb76 service cinder] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:41.571732 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d132d46d-ecc4-4209-8e9d-dd77217ebb76 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:41.611329 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d132d46d-ecc4-4209-8e9d-dd77217ebb76 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:41.612800 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d132d46d-ecc4-4209-8e9d-dd77217ebb76 service cinder] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:41.630618 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:40:41.692299 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 831/1673] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:40:41 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:40:41.975217 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3a1609-111a-4538-b685-64f24a80d666 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:42.061636 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d132d46d-ecc4-4209-8e9d-dd77217ebb76 service cinder] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:42.063655 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-d132d46d-ecc4-4209-8e9d-dd77217ebb76 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:42.262835 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d132d46d-ecc4-4209-8e9d-dd77217ebb76 service cinder] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:42.270345 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3a1609-111a-4538-b685-64f24a80d666 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:42.277303 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099ac58e-042a-43fe-af58-de5de045427b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:42.303874 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099ac58e-042a-43fe-af58-de5de045427b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:42.303874 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099ac58e-042a-43fe-af58-de5de045427b admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:42.303874 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099ac58e-042a-43fe-af58-de5de045427b admin admin] PATH_INFO: `/v3/projects/521a4e5109084dd2a7fb3fb73aa202af` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:42.303874 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099ac58e-042a-43fe-af58-de5de045427b admin admin] RBAC: Authorizing `identity:delete_project(project_id=521a4e5109084dd2a7fb3fb73aa202af)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:42.305419 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb57a9e-54fe-418e-92df-082798ccf34f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:42.305419 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb57a9e-54fe-418e-92df-082798ccf34f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:42.305419 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb57a9e-54fe-418e-92df-082798ccf34f admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:42.305419 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb57a9e-54fe-418e-92df-082798ccf34f admin admin] PATH_INFO: `/v3/users/b3e9b8a8ac12424f8cbf2e4fba9b4e28` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:42.305934 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099ac58e-042a-43fe-af58-de5de045427b admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:42.306131 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb57a9e-54fe-418e-92df-082798ccf34f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3e9b8a8ac12424f8cbf2e4fba9b4e28)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:42.306131 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb57a9e-54fe-418e-92df-082798ccf34f admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:42.856261 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-099ac58e-042a-43fe-af58-de5de045427b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '521a4e5109084dd2a7fb3fb73aa202af', 'request_id': 'req-099ac58e-042a-43fe-af58-de5de045427b'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:40:42.877410 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-099ac58e-042a-43fe-af58-de5de045427b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '521a4e5109084dd2a7fb3fb73aa202af', 'request_id': 'req-099ac58e-042a-43fe-af58-de5de045427b'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:40:42.884623 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-dbb57a9e-54fe-418e-92df-082798ccf34f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3e9b8a8ac12424f8cbf2e4fba9b4e28'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:40:42.886984 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-dbb57a9e-54fe-418e-92df-082798ccf34f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3e9b8a8ac12424f8cbf2e4fba9b4e28'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:40:42.888043 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-dbb57a9e-54fe-418e-92df-082798ccf34f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3e9b8a8ac12424f8cbf2e4fba9b4e28'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:40:42.994590 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-099ac58e-042a-43fe-af58-de5de045427b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '521a4e5109084dd2a7fb3fb73aa202af', 'request_id': 'req-099ac58e-042a-43fe-af58-de5de045427b'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:40:43.010880 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-099ac58e-042a-43fe-af58-de5de045427b admin admin] The token cache is being invalidate because project 521a4e5109084dd2a7fb3fb73aa202af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:40:43.011532 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-099ac58e-042a-43fe-af58-de5de045427b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:40:43.013009 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-099ac58e-042a-43fe-af58-de5de045427b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:40:43.015090 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 832/1674] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:40:42 2022] DELETE /identity/v3/projects/521a4e5109084dd2a7fb3fb73aa202af => generated 0 bytes in 954 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:40:43.018056 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfc6c97-b19e-4d78-ad3e-5f0cf7d3c00c None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:43.018633 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfc6c97-b19e-4d78-ad3e-5f0cf7d3c00c None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:43.019415 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfc6c97-b19e-4d78-ad3e-5f0cf7d3c00c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:43.069899 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-2cfc6c97-b19e-4d78-ad3e-5f0cf7d3c00c None None] MFA Rules not processed for user `8b9b29b53a7b490da29fbf6e5451c362`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:40:43.076293 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-2cfc6c97-b19e-4d78-ad3e-5f0cf7d3c00c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:43.112371 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-dbb57a9e-54fe-418e-92df-082798ccf34f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3e9b8a8ac12424f8cbf2e4fba9b4e28'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:40:43.145617 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-dbb57a9e-54fe-418e-92df-082798ccf34f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3e9b8a8ac12424f8cbf2e4fba9b4e28'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:40:43.186733 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 843/1675] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:40:41 2022] DELETE /identity/v3/users/b3e9b8a8ac12424f8cbf2e4fba9b4e28 => generated 0 bytes in 1212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:40:43.189274 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb57a9e-54fe-418e-92df-082798ccf34f admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:43.196166 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-dbb57a9e-54fe-418e-92df-082798ccf34f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:43.271898 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 833/1676] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:40:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:40:43.271898 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f301c9-f6d2-45e5-8734-b8485b6c109f None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:43.271898 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f301c9-f6d2-45e5-8734-b8485b6c109f None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:43.271898 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f301c9-f6d2-45e5-8734-b8485b6c109f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:43.323745 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-10f301c9-f6d2-45e5-8734-b8485b6c109f None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:40:43.326286 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-10f301c9-f6d2-45e5-8734-b8485b6c109f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:43.357791 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb57a9e-54fe-418e-92df-082798ccf34f admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:43.367518 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa4a5bd-ea2b-48e8-9458-b187b11bc8be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:43.368365 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa4a5bd-ea2b-48e8-9458-b187b11bc8be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:43.368966 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa4a5bd-ea2b-48e8-9458-b187b11bc8be admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:43.369370 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa4a5bd-ea2b-48e8-9458-b187b11bc8be admin admin] PATH_INFO: `/v3/projects/d0ec2d305f40451d804cccca8ae3ce57` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:43.370918 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa4a5bd-ea2b-48e8-9458-b187b11bc8be admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0ec2d305f40451d804cccca8ae3ce57)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:43.377226 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa4a5bd-ea2b-48e8-9458-b187b11bc8be admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:43.471435 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 834/1677] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:40:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:40:43.472940 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337aa9ce-13db-4177-9758-fbde790669da None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:43.473258 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337aa9ce-13db-4177-9758-fbde790669da None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:43.473530 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337aa9ce-13db-4177-9758-fbde790669da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:43.516779 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-337aa9ce-13db-4177-9758-fbde790669da None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:40:43.519312 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-337aa9ce-13db-4177-9758-fbde790669da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:43.566698 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 835/1678] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:40:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:40:43.567887 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337aa9ce-13db-4177-9758-fbde790669da None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:43.572986 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-337aa9ce-13db-4177-9758-fbde790669da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:43.608227 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-6aa4a5bd-ea2b-48e8-9458-b187b11bc8be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0ec2d305f40451d804cccca8ae3ce57', 'request_id': 'req-6aa4a5bd-ea2b-48e8-9458-b187b11bc8be'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:40:43.622855 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:40:43.821896 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-6aa4a5bd-ea2b-48e8-9458-b187b11bc8be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0ec2d305f40451d804cccca8ae3ce57', 'request_id': 'req-6aa4a5bd-ea2b-48e8-9458-b187b11bc8be'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:40:43.837054 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-6aa4a5bd-ea2b-48e8-9458-b187b11bc8be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0ec2d305f40451d804cccca8ae3ce57', 'request_id': 'req-6aa4a5bd-ea2b-48e8-9458-b187b11bc8be'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:40:43.844532 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337aa9ce-13db-4177-9758-fbde790669da None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:43.874437 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-6aa4a5bd-ea2b-48e8-9458-b187b11bc8be admin admin] The token cache is being invalidate because project d0ec2d305f40451d804cccca8ae3ce57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81723) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:40:43.874979 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-6aa4a5bd-ea2b-48e8-9458-b187b11bc8be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:40:43.877934 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-6aa4a5bd-ea2b-48e8-9458-b187b11bc8be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:40:43.881575 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 844/1679] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:40:43 2022] DELETE /identity/v3/projects/d0ec2d305f40451d804cccca8ae3ce57 => generated 0 bytes in 694 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:40:43.883754 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b720d535-ba1e-4fd8-b8b8-bf268e0330ce None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:43.888829 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b720d535-ba1e-4fd8-b8b8-bf268e0330ce None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:43.892826 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b720d535-ba1e-4fd8-b8b8-bf268e0330ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:43.967935 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-b720d535-ba1e-4fd8-b8b8-bf268e0330ce None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:40:43.969175 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-b720d535-ba1e-4fd8-b8b8-bf268e0330ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:44.158666 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fa62bd-ca5b-489f-8331-48475b72e80d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4f49c1b08c460f8ba6c451c9c92c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:44.159163 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fa62bd-ca5b-489f-8331-48475b72e80d service nova] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:44.159163 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fa62bd-ca5b-489f-8331-48475b72e80d service nova] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:44.159306 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fa62bd-ca5b-489f-8331-48475b72e80d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:44.169748 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 845/1680] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:40:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:40:44.170769 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b720d535-ba1e-4fd8-b8b8-bf268e0330ce None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:44.173447 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-b720d535-ba1e-4fd8-b8b8-bf268e0330ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:44.175138 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-96fa62bd-ca5b-489f-8331-48475b72e80d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:44.318964 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b720d535-ba1e-4fd8-b8b8-bf268e0330ce None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:44.334969 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d04a61e-e647-4d89-8850-6f4ee1bb763f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:44.335874 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d04a61e-e647-4d89-8850-6f4ee1bb763f service neutron] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:44.336367 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d04a61e-e647-4d89-8850-6f4ee1bb763f service neutron] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:44.337742 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d04a61e-e647-4d89-8850-6f4ee1bb763f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:44.347484 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-0d04a61e-e647-4d89-8850-6f4ee1bb763f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:44.388295 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d04a61e-e647-4d89-8850-6f4ee1bb763f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:44.390798 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d04a61e-e647-4d89-8850-6f4ee1bb763f service neutron] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:44.403531 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 846/1681] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:40:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:40:44.405322 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a313d5cc-d345-490d-8652-577158d7398d None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:44.405984 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a313d5cc-d345-490d-8652-577158d7398d None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:44.406716 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a313d5cc-d345-490d-8652-577158d7398d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:44.428643 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fa62bd-ca5b-489f-8331-48475b72e80d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:44.431658 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fa62bd-ca5b-489f-8331-48475b72e80d service nova] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:44.470934 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 836/1682] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:40:43 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 903 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:40:44.472107 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fa62bd-ca5b-489f-8331-48475b72e80d service nova] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:44.482048 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-a313d5cc-d345-490d-8652-577158d7398d None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:40:44.487183 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-a313d5cc-d345-490d-8652-577158d7398d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:44.529429 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fa62bd-ca5b-489f-8331-48475b72e80d service nova] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:44.554071 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 847/1683] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:40:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:40:44.555927 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207c5787-6add-492e-b66b-64a4fb78b8d6 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:44.560657 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e18325a-aef5-4d46-b43b-cc957273e53c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:44.561020 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207c5787-6add-492e-b66b-64a4fb78b8d6 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:44.561519 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207c5787-6add-492e-b66b-64a4fb78b8d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:44.567499 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e18325a-aef5-4d46-b43b-cc957273e53c admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:44.570798 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e18325a-aef5-4d46-b43b-cc957273e53c admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:44.575608 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e18325a-aef5-4d46-b43b-cc957273e53c admin admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:44.575608 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e18325a-aef5-4d46-b43b-cc957273e53c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:40:44.575608 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e18325a-aef5-4d46-b43b-cc957273e53c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:44.575608 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e18325a-aef5-4d46-b43b-cc957273e53c admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:44.633930 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 837/1684] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 24 06:40:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:40:44.641896 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e18325a-aef5-4d46-b43b-cc957273e53c admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:44.647896 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-207c5787-6add-492e-b66b-64a4fb78b8d6 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:40:44.657656 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-207c5787-6add-492e-b66b-64a4fb78b8d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:44.659239 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:40:44.682073 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e18325a-aef5-4d46-b43b-cc957273e53c admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:44.691233 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 848/1685] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:40:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:40:44.695868 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207c5787-6add-492e-b66b-64a4fb78b8d6 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:44.704569 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54923fd0-729b-4b49-a538-abc6f2f43f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:44.708699 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54923fd0-729b-4b49-a538-abc6f2f43f35 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:44.708699 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54923fd0-729b-4b49-a538-abc6f2f43f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:44.708699 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54923fd0-729b-4b49-a538-abc6f2f43f35 admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:44.709138 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54923fd0-729b-4b49-a538-abc6f2f43f35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:44.716326 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54923fd0-729b-4b49-a538-abc6f2f43f35 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:44.725016 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207c5787-6add-492e-b66b-64a4fb78b8d6 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:44.740168 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bd2c38-ed2b-4e10-b07b-c45e050d7586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:44.741010 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd2c38-ed2b-4e10-b07b-c45e050d7586 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:44.748598 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd2c38-ed2b-4e10-b07b-c45e050d7586 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:44.749295 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd2c38-ed2b-4e10-b07b-c45e050d7586 admin admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:44.753435 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bd2c38-ed2b-4e10-b07b-c45e050d7586 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:40:44.753435 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bd2c38-ed2b-4e10-b07b-c45e050d7586 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:44.754791 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bd2c38-ed2b-4e10-b07b-c45e050d7586 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:44.784682 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 849/1686] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 24 06:40:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:40:44.787406 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff75d7c-b58e-434b-b6ae-9c621a8110b7 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:44.788959 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff75d7c-b58e-434b-b6ae-9c621a8110b7 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:44.789510 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff75d7c-b58e-434b-b6ae-9c621a8110b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:44.883223 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-aff75d7c-b58e-434b-b6ae-9c621a8110b7 None None] MFA Rules not processed for user `68b76ae7f1aa41f2acf7e18790887bd5`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:40:44.884635 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-aff75d7c-b58e-434b-b6ae-9c621a8110b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:44.963953 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 838/1687] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:40:44 2022] POST /identity/v3/projects => generated 381 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:40:44.974160 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54923fd0-729b-4b49-a538-abc6f2f43f35 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:45.175610 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 850/1688] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:40:44 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 390 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:40:45.178489 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff75d7c-b58e-434b-b6ae-9c621a8110b7 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:45.241144 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54923fd0-729b-4b49-a538-abc6f2f43f35 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:45.244556 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff75d7c-b58e-434b-b6ae-9c621a8110b7 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:45.248648 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692684d6-ce69-45b6-8cad-18f9b643b20a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:45.249859 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692684d6-ce69-45b6-8cad-18f9b643b20a admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:45.251282 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692684d6-ce69-45b6-8cad-18f9b643b20a admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:45.252171 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692684d6-ce69-45b6-8cad-18f9b643b20a admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:45.254258 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692684d6-ce69-45b6-8cad-18f9b643b20a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:45.254955 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8177755-537c-4a8a-9d18-6c2030cb4e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:45.255601 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8177755-537c-4a8a-9d18-6c2030cb4e7c admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:45.255888 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8177755-537c-4a8a-9d18-6c2030cb4e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:45.256170 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8177755-537c-4a8a-9d18-6c2030cb4e7c admin admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:45.256810 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8177755-537c-4a8a-9d18-6c2030cb4e7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:45.259228 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8177755-537c-4a8a-9d18-6c2030cb4e7c admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:45.259673 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692684d6-ce69-45b6-8cad-18f9b643b20a admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:45.388594 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 839/1689] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:40:44 2022] POST /identity/v3/projects => generated 381 bytes in 424 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:40:45.394332 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed01b11f-99da-4fe2-97ef-ee3ba41c1f72 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:45.395308 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed01b11f-99da-4fe2-97ef-ee3ba41c1f72 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:45.396075 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed01b11f-99da-4fe2-97ef-ee3ba41c1f72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:45.427280 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 851/1690] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:40:45 2022] POST /identity/v3/users => generated 291 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:40:45.428654 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8177755-537c-4a8a-9d18-6c2030cb4e7c admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:45.452675 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8177755-537c-4a8a-9d18-6c2030cb4e7c admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:45.465972 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-ed01b11f-99da-4fe2-97ef-ee3ba41c1f72 None None] MFA Rules not processed for user `68b76ae7f1aa41f2acf7e18790887bd5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:40:45.472719 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-ed01b11f-99da-4fe2-97ef-ee3ba41c1f72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:45.602828 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d6222e-d53f-4ef8-8093-ba3708a222b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:45.603762 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d6222e-d53f-4ef8-8093-ba3708a222b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:45.604095 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d6222e-d53f-4ef8-8093-ba3708a222b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:45.604420 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d6222e-d53f-4ef8-8093-ba3708a222b3 admin admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:45.605069 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d6222e-d53f-4ef8-8093-ba3708a222b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:45.607387 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d6222e-d53f-4ef8-8093-ba3708a222b3 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:45.718296 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 840/1691] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:40:45 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:40:45.719942 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed01b11f-99da-4fe2-97ef-ee3ba41c1f72 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:45.766597 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed01b11f-99da-4fe2-97ef-ee3ba41c1f72 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:45.786045 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44924a19-2fc9-4860-9ee3-3113952d1322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:45.787065 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44924a19-2fc9-4860-9ee3-3113952d1322 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:45.787581 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44924a19-2fc9-4860-9ee3-3113952d1322 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:45.788112 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44924a19-2fc9-4860-9ee3-3113952d1322 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:45.790325 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44924a19-2fc9-4860-9ee3-3113952d1322 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:45.797867 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44924a19-2fc9-4860-9ee3-3113952d1322 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:45.802448 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 852/1692] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:40:45 2022] POST /identity/v3/users => generated 291 bytes in 374 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:40:45.803073 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d6222e-d53f-4ef8-8093-ba3708a222b3 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:45.808261 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-64d6222e-d53f-4ef8-8093-ba3708a222b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:45.825493 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 841/1693] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:40:45 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:40:45.826901 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44924a19-2fc9-4860-9ee3-3113952d1322 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:45.829367 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-44924a19-2fc9-4860-9ee3-3113952d1322 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:45.863696 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44924a19-2fc9-4860-9ee3-3113952d1322 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:45.871487 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9322e30e-891c-47f4-8d8d-9024ac89ff5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:45.873050 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9322e30e-891c-47f4-8d8d-9024ac89ff5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:45.873050 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9322e30e-891c-47f4-8d8d-9024ac89ff5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:45.873050 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9322e30e-891c-47f4-8d8d-9024ac89ff5e admin admin] PATH_INFO: `/v3/projects/0e2df09f149a4902be90ebd39ff1988a` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:45.874951 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9322e30e-891c-47f4-8d8d-9024ac89ff5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e2df09f149a4902be90ebd39ff1988a)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:45.877554 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9322e30e-891c-47f4-8d8d-9024ac89ff5e admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:45.976394 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d6222e-d53f-4ef8-8093-ba3708a222b3 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:45.982946 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27b115c-9f8a-48f6-9c44-ac7b72075a73 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4f49c1b08c460f8ba6c451c9c92c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:45.983785 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b115c-9f8a-48f6-9c44-ac7b72075a73 service nova] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:45.984071 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b115c-9f8a-48f6-9c44-ac7b72075a73 service nova] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:45.984355 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b115c-9f8a-48f6-9c44-ac7b72075a73 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:45.995097 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27b115c-9f8a-48f6-9c44-ac7b72075a73 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:45.996976 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27b115c-9f8a-48f6-9c44-ac7b72075a73 service nova] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:46.074583 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 853/1694] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:40:45 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:40:46.079793 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27b115c-9f8a-48f6-9c44-ac7b72075a73 service nova] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:46.116277 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-9322e30e-891c-47f4-8d8d-9024ac89ff5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e2df09f149a4902be90ebd39ff1988a', 'request_id': 'req-9322e30e-891c-47f4-8d8d-9024ac89ff5e'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:40:46.118411 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27b115c-9f8a-48f6-9c44-ac7b72075a73 service nova] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:46.144311 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-9322e30e-891c-47f4-8d8d-9024ac89ff5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e2df09f149a4902be90ebd39ff1988a', 'request_id': 'req-9322e30e-891c-47f4-8d8d-9024ac89ff5e'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:40:46.256945 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-9322e30e-891c-47f4-8d8d-9024ac89ff5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e2df09f149a4902be90ebd39ff1988a', 'request_id': 'req-9322e30e-891c-47f4-8d8d-9024ac89ff5e'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:40:46.286508 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-9322e30e-891c-47f4-8d8d-9024ac89ff5e admin admin] The token cache is being invalidate because project 0e2df09f149a4902be90ebd39ff1988a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:40:46.289020 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-9322e30e-891c-47f4-8d8d-9024ac89ff5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:40:46.294905 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-9322e30e-891c-47f4-8d8d-9024ac89ff5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:40:46.306112 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 842/1695] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:40:45 2022] DELETE /identity/v3/projects/0e2df09f149a4902be90ebd39ff1988a => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:40:46.313810 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9322e30e-891c-47f4-8d8d-9024ac89ff5e admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:46.344162 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-9322e30e-891c-47f4-8d8d-9024ac89ff5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:46.373519 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9322e30e-891c-47f4-8d8d-9024ac89ff5e admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:46.379641 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7415282e-fcfc-44b8-bc36-ab3a1bc8dd20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:46.384542 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7415282e-fcfc-44b8-bc36-ab3a1bc8dd20 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:46.384884 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7415282e-fcfc-44b8-bc36-ab3a1bc8dd20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:46.385160 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7415282e-fcfc-44b8-bc36-ab3a1bc8dd20 admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:46.385903 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7415282e-fcfc-44b8-bc36-ab3a1bc8dd20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:46.387659 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7415282e-fcfc-44b8-bc36-ab3a1bc8dd20 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:46.408459 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d709dd-5d1e-474f-88e1-d7425804d89e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:46.411601 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d709dd-5d1e-474f-88e1-d7425804d89e admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:46.412303 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d709dd-5d1e-474f-88e1-d7425804d89e admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:46.441338 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d709dd-5d1e-474f-88e1-d7425804d89e admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:46.441338 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d709dd-5d1e-474f-88e1-d7425804d89e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:46.441338 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d709dd-5d1e-474f-88e1-d7425804d89e admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:46.441842 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 854/1696] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:40:46 2022] GET /identity/v3/roles => generated 1420 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:40:46.455879 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7415282e-fcfc-44b8-bc36-ab3a1bc8dd20 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:46.464034 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-7415282e-fcfc-44b8-bc36-ab3a1bc8dd20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:46.473099 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 843/1697] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:40:46 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:40:46.474870 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d709dd-5d1e-474f-88e1-d7425804d89e admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:46.487421 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-b4d709dd-5d1e-474f-88e1-d7425804d89e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:46.513166 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7415282e-fcfc-44b8-bc36-ab3a1bc8dd20 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:46.520854 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ce7c4b-a299-41ab-92d6-fb2d5b6361e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:46.531171 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ce7c4b-a299-41ab-92d6-fb2d5b6361e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:46.531171 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ce7c4b-a299-41ab-92d6-fb2d5b6361e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:46.532565 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ce7c4b-a299-41ab-92d6-fb2d5b6361e9 admin admin] PATH_INFO: `/v3/users/005e9efcb6ef4102ad5493a5c666554c` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:46.539593 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ce7c4b-a299-41ab-92d6-fb2d5b6361e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=005e9efcb6ef4102ad5493a5c666554c)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:46.542033 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ce7c4b-a299-41ab-92d6-fb2d5b6361e9 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:46.582802 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d709dd-5d1e-474f-88e1-d7425804d89e admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:46.604925 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b54e926-fe67-4196-97f1-fa4ee0886886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:46.606360 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b54e926-fe67-4196-97f1-fa4ee0886886 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:46.607043 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b54e926-fe67-4196-97f1-fa4ee0886886 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:46.607791 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b54e926-fe67-4196-97f1-fa4ee0886886 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:46.609201 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b54e926-fe67-4196-97f1-fa4ee0886886 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:46.613526 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b54e926-fe67-4196-97f1-fa4ee0886886 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:46.648495 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 844/1698] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:40:46 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:40:46.651448 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b54e926-fe67-4196-97f1-fa4ee0886886 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:46.670476 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b54e926-fe67-4196-97f1-fa4ee0886886 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:46.685334 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbf92a7-aa17-4766-910d-a38fa0ef6689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:46.688921 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbf92a7-aa17-4766-910d-a38fa0ef6689 admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:46.689810 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbf92a7-aa17-4766-910d-a38fa0ef6689 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:46.690571 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbf92a7-aa17-4766-910d-a38fa0ef6689 admin admin] PATH_INFO: `/v3/projects/324e3f0a469f48f3943ecfc9ee9c5749/users/bbbd204d171e415ebd1b928e00d3303f/roles/e1f9b167c51c4a388cbc9594db8a465d` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:46.738659 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbf92a7-aa17-4766-910d-a38fa0ef6689 admin admin] RBAC: Authorizing `identity:create_grant(project_id=324e3f0a469f48f3943ecfc9ee9c5749, user_id=bbbd204d171e415ebd1b928e00d3303f, role_id=e1f9b167c51c4a388cbc9594db8a465d)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:46.759670 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbf92a7-aa17-4766-910d-a38fa0ef6689 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:46.831740 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:40:46.914867 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 845/1699] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:40:46 2022] PUT /identity/v3/projects/324e3f0a469f48f3943ecfc9ee9c5749/users/bbbd204d171e415ebd1b928e00d3303f/roles/e1f9b167c51c4a388cbc9594db8a465d => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:40:46.920400 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbf92a7-aa17-4766-910d-a38fa0ef6689 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:46.974514 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-85ce7c4b-a299-41ab-92d6-fb2d5b6361e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '005e9efcb6ef4102ad5493a5c666554c'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:40:46.979628 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-85ce7c4b-a299-41ab-92d6-fb2d5b6361e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '005e9efcb6ef4102ad5493a5c666554c'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:40:46.984912 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-85ce7c4b-a299-41ab-92d6-fb2d5b6361e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '005e9efcb6ef4102ad5493a5c666554c'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:40:47.130405 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-85ce7c4b-a299-41ab-92d6-fb2d5b6361e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '005e9efcb6ef4102ad5493a5c666554c'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:40:47.152055 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-85ce7c4b-a299-41ab-92d6-fb2d5b6361e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '005e9efcb6ef4102ad5493a5c666554c'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:40:47.204237 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 855/1700] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:40:46 2022] DELETE /identity/v3/users/005e9efcb6ef4102ad5493a5c666554c => generated 0 bytes in 762 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:40:47.206868 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0725f4f-ac02-4391-99fa-48c22bb16d4a None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:47.207590 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0725f4f-ac02-4391-99fa-48c22bb16d4a None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:47.213887 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0725f4f-ac02-4391-99fa-48c22bb16d4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:47.330759 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbf92a7-aa17-4766-910d-a38fa0ef6689 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:47.333487 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-f0725f4f-ac02-4391-99fa-48c22bb16d4a None None] MFA Rules not processed for user `bbbd204d171e415ebd1b928e00d3303f`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:40:47.334794 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-f0725f4f-ac02-4391-99fa-48c22bb16d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:47.437148 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 856/1701] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:40:47 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:40:47.441783 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0725f4f-ac02-4391-99fa-48c22bb16d4a None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:47.443599 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-f0725f4f-ac02-4391-99fa-48c22bb16d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:47.579217 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563e9dd2-5df5-40a5-a604-61b624a594bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:47.586352 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563e9dd2-5df5-40a5-a604-61b624a594bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:47.586786 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563e9dd2-5df5-40a5-a604-61b624a594bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:47.586957 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0725f4f-ac02-4391-99fa-48c22bb16d4a None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:47.587131 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563e9dd2-5df5-40a5-a604-61b624a594bd admin admin] PATH_INFO: `/v3/projects/50e9d12bda7240edb650f24518264ef9/users/dc2801778ce74ada8dc8aa4f51526920/roles/e1f9b167c51c4a388cbc9594db8a465d` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:47.599715 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fb664a-859c-43ea-b36f-c542cb9eb301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:47.600359 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb664a-859c-43ea-b36f-c542cb9eb301 admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:47.600640 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb664a-859c-43ea-b36f-c542cb9eb301 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:47.600910 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb664a-859c-43ea-b36f-c542cb9eb301 admin admin] PATH_INFO: `/v3/projects` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:47.605043 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fb664a-859c-43ea-b36f-c542cb9eb301 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:47.613287 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fb664a-859c-43ea-b36f-c542cb9eb301 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:47.617625 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563e9dd2-5df5-40a5-a604-61b624a594bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=50e9d12bda7240edb650f24518264ef9, user_id=dc2801778ce74ada8dc8aa4f51526920, role_id=e1f9b167c51c4a388cbc9594db8a465d)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:47.628184 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563e9dd2-5df5-40a5-a604-61b624a594bd admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:47.857171 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 857/1702] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:40:47 2022] POST /identity/v3/projects => generated 381 bytes in 416 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:40:47.858846 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e06eab0-540b-4cee-9950-8918e387fc7b None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:47.859146 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e06eab0-540b-4cee-9950-8918e387fc7b None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:47.859442 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e06eab0-540b-4cee-9950-8918e387fc7b None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:47.861015 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 858/1703] 10.222.0.47 () {54 vars in 912 bytes} [Wed Aug 24 06:40:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:40:47.863824 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e06eab0-540b-4cee-9950-8918e387fc7b None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:47.870376 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 846/1704] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:40:46 2022] PUT /identity/v3/projects/50e9d12bda7240edb650f24518264ef9/users/dc2801778ce74ada8dc8aa4f51526920/roles/e1f9b167c51c4a388cbc9594db8a465d => generated 0 bytes in 955 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:40:47.904513 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e487ebd-6e18-43d8-b5b4-46787a8df952 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:47.910955 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e487ebd-6e18-43d8-b5b4-46787a8df952 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:47.911855 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e487ebd-6e18-43d8-b5b4-46787a8df952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:47.980722 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:40:48.059308 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-7e487ebd-6e18-43d8-b5b4-46787a8df952 None None] MFA Rules not processed for user `dc2801778ce74ada8dc8aa4f51526920`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:40:48.060503 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-7e487ebd-6e18-43d8-b5b4-46787a8df952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:48.096775 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e06eab0-540b-4cee-9950-8918e387fc7b None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:48.117784 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a29d8ee-1609-4e77-bfa1-7f8b7c3c9a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:48.118703 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a29d8ee-1609-4e77-bfa1-7f8b7c3c9a27 admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:48.118992 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a29d8ee-1609-4e77-bfa1-7f8b7c3c9a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:48.119277 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a29d8ee-1609-4e77-bfa1-7f8b7c3c9a27 admin admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:48.119922 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a29d8ee-1609-4e77-bfa1-7f8b7c3c9a27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:48.121971 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a29d8ee-1609-4e77-bfa1-7f8b7c3c9a27 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:48.223932 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 847/1705] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:40:47 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:40:48.225891 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feb7609-a49f-46a1-8773-429b1a0531ef None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:48.226345 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feb7609-a49f-46a1-8773-429b1a0531ef None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:48.226928 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feb7609-a49f-46a1-8773-429b1a0531ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:48.260617 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 859/1706] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:40:47 2022] POST /identity/v3/users => generated 297 bytes in 397 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:40:48.261381 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a29d8ee-1609-4e77-bfa1-7f8b7c3c9a27 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:48.267596 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-4a29d8ee-1609-4e77-bfa1-7f8b7c3c9a27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:48.285407 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-3feb7609-a49f-46a1-8773-429b1a0531ef None None] MFA Rules not processed for user `f0ba3f03ec15475ab6ebf64994ad742c`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:40:48.287503 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-3feb7609-a49f-46a1-8773-429b1a0531ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:48.298194 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a29d8ee-1609-4e77-bfa1-7f8b7c3c9a27 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:48.315265 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48983dc5-4bb9-4c77-9bb0-78891d8d8c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:48.316154 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48983dc5-4bb9-4c77-9bb0-78891d8d8c45 admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:48.316601 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48983dc5-4bb9-4c77-9bb0-78891d8d8c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:48.317039 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48983dc5-4bb9-4c77-9bb0-78891d8d8c45 admin admin] PATH_INFO: `/v3/projects` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:48.317868 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48983dc5-4bb9-4c77-9bb0-78891d8d8c45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:48.321284 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48983dc5-4bb9-4c77-9bb0-78891d8d8c45 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:48.365929 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 848/1707] 10.222.0.47 () {58 vars in 1009 bytes} [Wed Aug 24 06:40:48 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:40:48.367022 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feb7609-a49f-46a1-8773-429b1a0531ef None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:48.383661 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feb7609-a49f-46a1-8773-429b1a0531ef None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:48.384737 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3a0a31-901d-4a04-b4d1-8ae6ba82a412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:48.385569 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3a0a31-901d-4a04-b4d1-8ae6ba82a412 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:48.385678 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3a0a31-901d-4a04-b4d1-8ae6ba82a412 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:48.385935 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3a0a31-901d-4a04-b4d1-8ae6ba82a412 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:48.386688 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3a0a31-901d-4a04-b4d1-8ae6ba82a412 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:48.389075 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3a0a31-901d-4a04-b4d1-8ae6ba82a412 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:48.404253 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 849/1708] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:40:48 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:40:48.404253 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3a0a31-901d-4a04-b4d1-8ae6ba82a412 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:48.405789 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-1f3a0a31-901d-4a04-b4d1-8ae6ba82a412 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:48.438419 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3a0a31-901d-4a04-b4d1-8ae6ba82a412 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:48.445472 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a6d118-bfc4-46e2-8208-d4c67bcff41c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:48.446549 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a6d118-bfc4-46e2-8208-d4c67bcff41c service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:48.446843 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a6d118-bfc4-46e2-8208-d4c67bcff41c service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:48.447489 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a6d118-bfc4-46e2-8208-d4c67bcff41c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:48.458517 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a6d118-bfc4-46e2-8208-d4c67bcff41c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:48.460029 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a6d118-bfc4-46e2-8208-d4c67bcff41c service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:48.471409 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 850/1709] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:40:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:40:48.472459 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a6d118-bfc4-46e2-8208-d4c67bcff41c service neutron] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:48.485899 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a6d118-bfc4-46e2-8208-d4c67bcff41c service neutron] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:48.492150 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddd4f62-b0f0-46fe-96c3-6b37cf03c354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:48.492929 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddd4f62-b0f0-46fe-96c3-6b37cf03c354 admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:48.493278 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddd4f62-b0f0-46fe-96c3-6b37cf03c354 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:48.493577 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddd4f62-b0f0-46fe-96c3-6b37cf03c354 admin admin] PATH_INFO: `/v3/projects/33f9a38265ce4d65aa35b54e01a2a906/users/1abd7b7f2cd54f1b8bfe68807041afe7/roles/486144ad1bcd417e86acdbc1d14327bf` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:48.510054 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 860/1710] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:40:48 2022] POST /identity/v3/projects => generated 379 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:40:48.514991 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48983dc5-4bb9-4c77-9bb0-78891d8d8c45 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:48.520008 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddd4f62-b0f0-46fe-96c3-6b37cf03c354 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33f9a38265ce4d65aa35b54e01a2a906, user_id=1abd7b7f2cd54f1b8bfe68807041afe7, role_id=486144ad1bcd417e86acdbc1d14327bf)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:48.523669 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddd4f62-b0f0-46fe-96c3-6b37cf03c354 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:48.684683 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 851/1711] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:40:48 2022] PUT /identity/v3/projects/33f9a38265ce4d65aa35b54e01a2a906/users/1abd7b7f2cd54f1b8bfe68807041afe7/roles/486144ad1bcd417e86acdbc1d14327bf => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:40:48.690388 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddd4f62-b0f0-46fe-96c3-6b37cf03c354 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:48.712975 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48983dc5-4bb9-4c77-9bb0-78891d8d8c45 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:48.808805 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddd4f62-b0f0-46fe-96c3-6b37cf03c354 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:48.822764 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb06645-25bc-4039-b8ff-973f6db30656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:48.824089 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb06645-25bc-4039-b8ff-973f6db30656 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:48.825872 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb06645-25bc-4039-b8ff-973f6db30656 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:48.826441 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb06645-25bc-4039-b8ff-973f6db30656 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:48.827449 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb06645-25bc-4039-b8ff-973f6db30656 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:48.852264 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb06645-25bc-4039-b8ff-973f6db30656 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:48.900908 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 852/1712] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:40:48 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:40:48.903772 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb06645-25bc-4039-b8ff-973f6db30656 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:48.936793 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb06645-25bc-4039-b8ff-973f6db30656 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:48.959928 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6187bb82-7107-4873-9ae1-bc3f2f1c9b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:48.960628 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6187bb82-7107-4873-9ae1-bc3f2f1c9b1f admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:48.960870 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20b070f-e33b-4297-a5bf-defe2bf3809f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:48.961037 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6187bb82-7107-4873-9ae1-bc3f2f1c9b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:48.961153 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6187bb82-7107-4873-9ae1-bc3f2f1c9b1f admin admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:48.961929 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6187bb82-7107-4873-9ae1-bc3f2f1c9b1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:48.962184 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20b070f-e33b-4297-a5bf-defe2bf3809f admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:48.962439 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20b070f-e33b-4297-a5bf-defe2bf3809f admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:48.962740 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20b070f-e33b-4297-a5bf-defe2bf3809f admin admin] PATH_INFO: `/v3/projects/33f9a38265ce4d65aa35b54e01a2a906/users/1abd7b7f2cd54f1b8bfe68807041afe7/roles/486144ad1bcd417e86acdbc1d14327bf` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:48.963994 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6187bb82-7107-4873-9ae1-bc3f2f1c9b1f admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:48.965941 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20b070f-e33b-4297-a5bf-defe2bf3809f admin admin] RBAC: Authorizing `identity:create_grant(project_id=33f9a38265ce4d65aa35b54e01a2a906, user_id=1abd7b7f2cd54f1b8bfe68807041afe7, role_id=486144ad1bcd417e86acdbc1d14327bf)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:48.970499 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20b070f-e33b-4297-a5bf-defe2bf3809f admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:48.989666 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 853/1713] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:40:48 2022] PUT /identity/v3/projects/33f9a38265ce4d65aa35b54e01a2a906/users/1abd7b7f2cd54f1b8bfe68807041afe7/roles/486144ad1bcd417e86acdbc1d14327bf => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:40:49.007405 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3880f63f-f78f-426c-9ec8-47a8c822db2c None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:49.008311 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3880f63f-f78f-426c-9ec8-47a8c822db2c None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:49.008627 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3880f63f-f78f-426c-9ec8-47a8c822db2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:49.064844 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:40:49.100121 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-3880f63f-f78f-426c-9ec8-47a8c822db2c None None] MFA Rules not processed for user `1abd7b7f2cd54f1b8bfe68807041afe7`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:40:49.104224 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-3880f63f-f78f-426c-9ec8-47a8c822db2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:49.119352 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 861/1714] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:40:48 2022] POST /identity/v3/users => generated 296 bytes in 605 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:40:49.123988 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6187bb82-7107-4873-9ae1-bc3f2f1c9b1f admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:49.174591 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6187bb82-7107-4873-9ae1-bc3f2f1c9b1f admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:49.192555 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:40:49.367143 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 854/1715] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:40:49 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:40:49.367143 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3880f63f-f78f-426c-9ec8-47a8c822db2c None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:49.367143 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-3880f63f-f78f-426c-9ec8-47a8c822db2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:49.411695 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab3f449-bb20-4e9b-a437-6009ea656ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:49.412674 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab3f449-bb20-4e9b-a437-6009ea656ba3 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:49.413105 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab3f449-bb20-4e9b-a437-6009ea656ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:49.413585 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab3f449-bb20-4e9b-a437-6009ea656ba3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:49.414544 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab3f449-bb20-4e9b-a437-6009ea656ba3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:49.416459 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab3f449-bb20-4e9b-a437-6009ea656ba3 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:49.469896 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 862/1716] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:40:49 2022] GET /identity/v3/roles => generated 1420 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:40:49.474977 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab3f449-bb20-4e9b-a437-6009ea656ba3 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:49.517266 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab3f449-bb20-4e9b-a437-6009ea656ba3 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:49.524767 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3880f63f-f78f-426c-9ec8-47a8c822db2c None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:49.528492 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6722f1-68e5-41a9-afe6-3eb97ee1ca39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:49.529828 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6722f1-68e5-41a9-afe6-3eb97ee1ca39 admin admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:49.530805 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6722f1-68e5-41a9-afe6-3eb97ee1ca39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:49.531929 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6722f1-68e5-41a9-afe6-3eb97ee1ca39 admin admin] PATH_INFO: `/v3/projects/fa545d11e3f04f2488f53e6f7d5873b8/users/9a224fd5432946e3be47435a42434bd6/roles/486144ad1bcd417e86acdbc1d14327bf` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:49.571421 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e463964-a2df-4639-9e75-44bc66d60368 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36639dce4f97412f89d71367f650b901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:49.571421 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e463964-a2df-4639-9e75-44bc66d60368 service cinder] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:49.572881 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e463964-a2df-4639-9e75-44bc66d60368 service cinder] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:49.573564 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e463964-a2df-4639-9e75-44bc66d60368 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:49.590123 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e463964-a2df-4639-9e75-44bc66d60368 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:49.593002 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e463964-a2df-4639-9e75-44bc66d60368 service cinder] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:49.606866 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6722f1-68e5-41a9-afe6-3eb97ee1ca39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa545d11e3f04f2488f53e6f7d5873b8, user_id=9a224fd5432946e3be47435a42434bd6, role_id=486144ad1bcd417e86acdbc1d14327bf)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:49.611963 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 855/1717] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:40:49 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:40:49.616378 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6722f1-68e5-41a9-afe6-3eb97ee1ca39 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:49.769206 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e463964-a2df-4639-9e75-44bc66d60368 service cinder] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:49.769206 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-6e463964-a2df-4639-9e75-44bc66d60368 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:49.821081 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 863/1718] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:40:49 2022] PUT /identity/v3/projects/fa545d11e3f04f2488f53e6f7d5873b8/users/9a224fd5432946e3be47435a42434bd6/roles/486144ad1bcd417e86acdbc1d14327bf => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:40:49.822058 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6722f1-68e5-41a9-afe6-3eb97ee1ca39 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:49.839952 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e463964-a2df-4639-9e75-44bc66d60368 service cinder] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:49.993283 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6722f1-68e5-41a9-afe6-3eb97ee1ca39 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:50.000669 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93656fc5-4c97-4442-8559-1a8deac3f5a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:50.001366 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93656fc5-4c97-4442-8559-1a8deac3f5a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:50.003674 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc021bef-3b34-4691-817f-6328fd7f5e51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36639dce4f97412f89d71367f650b901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:50.004858 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc021bef-3b34-4691-817f-6328fd7f5e51 service cinder] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:50.005378 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc021bef-3b34-4691-817f-6328fd7f5e51 service cinder] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:50.005872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc021bef-3b34-4691-817f-6328fd7f5e51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:50.006791 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93656fc5-4c97-4442-8559-1a8deac3f5a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:50.007147 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93656fc5-4c97-4442-8559-1a8deac3f5a8 admin admin] PATH_INFO: `/v3/projects/718912837e2e43a984ca5b5437ff4ce8` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:50.009421 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93656fc5-4c97-4442-8559-1a8deac3f5a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=718912837e2e43a984ca5b5437ff4ce8)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:50.011390 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93656fc5-4c97-4442-8559-1a8deac3f5a8 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:50.183981 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc021bef-3b34-4691-817f-6328fd7f5e51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:50.189475 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc021bef-3b34-4691-817f-6328fd7f5e51 service cinder] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:50.214639 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 864/1719] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:40:49 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:40:50.219668 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc021bef-3b34-4691-817f-6328fd7f5e51 service cinder] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:50.269878 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-93656fc5-4c97-4442-8559-1a8deac3f5a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '718912837e2e43a984ca5b5437ff4ce8', 'request_id': 'req-93656fc5-4c97-4442-8559-1a8deac3f5a8'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:40:50.272192 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc021bef-3b34-4691-817f-6328fd7f5e51 service cinder] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:50.292626 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-93656fc5-4c97-4442-8559-1a8deac3f5a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '718912837e2e43a984ca5b5437ff4ce8', 'request_id': 'req-93656fc5-4c97-4442-8559-1a8deac3f5a8'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:40:50.393575 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-93656fc5-4c97-4442-8559-1a8deac3f5a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '718912837e2e43a984ca5b5437ff4ce8', 'request_id': 'req-93656fc5-4c97-4442-8559-1a8deac3f5a8'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:40:50.418511 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-93656fc5-4c97-4442-8559-1a8deac3f5a8 admin admin] The token cache is being invalidate because project 718912837e2e43a984ca5b5437ff4ce8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:40:50.420717 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-93656fc5-4c97-4442-8559-1a8deac3f5a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:40:50.425015 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-93656fc5-4c97-4442-8559-1a8deac3f5a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:40:50.427201 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 856/1720] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:40:49 2022] DELETE /identity/v3/projects/718912837e2e43a984ca5b5437ff4ce8 => generated 0 bytes in 667 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:40:50.433740 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d677d98-699b-4da1-b8d4-eaf471178439 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:50.434913 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d677d98-699b-4da1-b8d4-eaf471178439 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:50.436744 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d677d98-699b-4da1-b8d4-eaf471178439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:50.479951 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d391c1-29e5-4dad-a870-703d15c2446f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:50.488003 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d391c1-29e5-4dad-a870-703d15c2446f admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:50.488432 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d391c1-29e5-4dad-a870-703d15c2446f admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:50.488938 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d391c1-29e5-4dad-a870-703d15c2446f admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:50.491758 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d391c1-29e5-4dad-a870-703d15c2446f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:50.498695 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d391c1-29e5-4dad-a870-703d15c2446f admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:50.536486 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 865/1721] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:40:50 2022] GET /identity/v3/roles => generated 1420 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:40:50.539352 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3211be8-de33-449e-b98a-21ab3938b08b None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:50.539913 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3211be8-de33-449e-b98a-21ab3938b08b None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:50.540200 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3211be8-de33-449e-b98a-21ab3938b08b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:50.555616 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-5d677d98-699b-4da1-b8d4-eaf471178439 None None] MFA Rules not processed for user `52f796d5013d4a31b107fb86c5523c9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:40:50.557056 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-5d677d98-699b-4da1-b8d4-eaf471178439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:50.576516 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-f3211be8-de33-449e-b98a-21ab3938b08b None None] MFA Rules not processed for user `02ad6cdc4ffe4917a7bbf6814297f980`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:40:50.577810 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-f3211be8-de33-449e-b98a-21ab3938b08b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:50.679692 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:40:50.728733 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:40:50.736872 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 857/1722] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:40:50 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:40:50.741513 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60400545-8901-4652-92e5-254d3520c763 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:50.743793 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60400545-8901-4652-92e5-254d3520c763 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:50.744175 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60400545-8901-4652-92e5-254d3520c763 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:50.757516 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 858/1723] 10.222.0.47 () {54 vars in 961 bytes} [Wed Aug 24 06:40:50 2022] GET /identity => generated 270 bytes in 20 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:40:50.760826 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e014c2cd-2c8c-41da-98a9-cca4a6ec5392 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:50.761174 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e014c2cd-2c8c-41da-98a9-cca4a6ec5392 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:50.762494 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e014c2cd-2c8c-41da-98a9-cca4a6ec5392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:50.768293 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 866/1724] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:40:50 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:40:50.769568 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3211be8-de33-449e-b98a-21ab3938b08b None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:50.771354 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-f3211be8-de33-449e-b98a-21ab3938b08b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:50.797560 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-e014c2cd-2c8c-41da-98a9-cca4a6ec5392 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:40:50.800926 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-e014c2cd-2c8c-41da-98a9-cca4a6ec5392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:50.801781 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3211be8-de33-449e-b98a-21ab3938b08b None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:50.809588 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b6cf77-deba-47cd-a65b-37118de5bdcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:50.810505 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b6cf77-deba-47cd-a65b-37118de5bdcd admin admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:50.811885 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b6cf77-deba-47cd-a65b-37118de5bdcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:50.812393 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b6cf77-deba-47cd-a65b-37118de5bdcd admin admin] PATH_INFO: `/v3/projects/fa545d11e3f04f2488f53e6f7d5873b8/users/9a224fd5432946e3be47435a42434bd6/roles/486144ad1bcd417e86acdbc1d14327bf` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:50.812979 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 859/1725] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:40:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:40:50.814561 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008f704f-e307-4025-88ff-91359ffa5227 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:50.815177 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008f704f-e307-4025-88ff-91359ffa5227 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:50.815550 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008f704f-e307-4025-88ff-91359ffa5227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:50.817702 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b6cf77-deba-47cd-a65b-37118de5bdcd admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa545d11e3f04f2488f53e6f7d5873b8, user_id=9a224fd5432946e3be47435a42434bd6, role_id=486144ad1bcd417e86acdbc1d14327bf)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:50.825604 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b6cf77-deba-47cd-a65b-37118de5bdcd admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:50.848708 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-008f704f-e307-4025-88ff-91359ffa5227 None None] MFA Rules not processed for user `306188d42dbd4b5b84082ab3842eaf9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:40:50.850252 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-008f704f-e307-4025-88ff-91359ffa5227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:50.867414 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 867/1726] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:40:50 2022] PUT /identity/v3/projects/fa545d11e3f04f2488f53e6f7d5873b8/users/9a224fd5432946e3be47435a42434bd6/roles/486144ad1bcd417e86acdbc1d14327bf => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:40:50.878954 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b6cf77-deba-47cd-a65b-37118de5bdcd admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:50.883374 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-53b6cf77-deba-47cd-a65b-37118de5bdcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:51.054128 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b6cf77-deba-47cd-a65b-37118de5bdcd admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:51.076441 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de24ac8-6ad4-4284-973c-39e184dddbd8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4f49c1b08c460f8ba6c451c9c92c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:51.077303 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de24ac8-6ad4-4284-973c-39e184dddbd8 service nova] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:51.077918 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de24ac8-6ad4-4284-973c-39e184dddbd8 service nova] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:51.078536 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de24ac8-6ad4-4284-973c-39e184dddbd8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:51.097991 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 860/1727] 10.222.0.47 () {58 vars in 1058 bytes} [Wed Aug 24 06:40:50 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:40:51.099534 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008f704f-e307-4025-88ff-91359ffa5227 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:51.102016 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-008f704f-e307-4025-88ff-91359ffa5227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:51.130583 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008f704f-e307-4025-88ff-91359ffa5227 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:51.145390 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ef90d3-1023-4868-9aa5-85d38695fd3e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4f49c1b08c460f8ba6c451c9c92c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:51.146418 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ef90d3-1023-4868-9aa5-85d38695fd3e service nova] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:51.146903 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ef90d3-1023-4868-9aa5-85d38695fd3e service nova] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:51.147405 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ef90d3-1023-4868-9aa5-85d38695fd3e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:51.230565 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ef90d3-1023-4868-9aa5-85d38695fd3e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:51.232554 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ef90d3-1023-4868-9aa5-85d38695fd3e service nova] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:51.252671 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de24ac8-6ad4-4284-973c-39e184dddbd8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:51.254497 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de24ac8-6ad4-4284-973c-39e184dddbd8 service nova] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:51.259068 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 861/1728] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:40:51 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:40:51.269924 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 868/1729] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:40:50 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:40:51.271659 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae38ad9-77c5-444e-9c94-91800afe3c01 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:51.271997 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae38ad9-77c5-444e-9c94-91800afe3c01 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:51.272426 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae38ad9-77c5-444e-9c94-91800afe3c01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:51.291782 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcab8bc-cb4b-441d-af78-a2d276d561a7 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:51.293027 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcab8bc-cb4b-441d-af78-a2d276d561a7 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:51.295267 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcab8bc-cb4b-441d-af78-a2d276d561a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:51.304128 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-7ae38ad9-77c5-444e-9c94-91800afe3c01 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:40:51.307977 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-7ae38ad9-77c5-444e-9c94-91800afe3c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:51.445400 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-8bcab8bc-cb4b-441d-af78-a2d276d561a7 None None] MFA Rules not processed for user `9a224fd5432946e3be47435a42434bd6`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:40:51.449592 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-8bcab8bc-cb4b-441d-af78-a2d276d561a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:51.502464 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 869/1730] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:40:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:40:51.507640 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae38ad9-77c5-444e-9c94-91800afe3c01 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:51.584710 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae38ad9-77c5-444e-9c94-91800afe3c01 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:51.616572 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f6cac6-845b-476c-9833-b06c2afe8b8b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '306188d42dbd4b5b84082ab3842eaf9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:51.619648 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f6cac6-845b-476c-9833-b06c2afe8b8b service placement] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:51.619648 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f6cac6-845b-476c-9833-b06c2afe8b8b service placement] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:51.619648 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f6cac6-845b-476c-9833-b06c2afe8b8b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:51.644058 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-48f6cac6-845b-476c-9833-b06c2afe8b8b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:51.704792 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f6cac6-845b-476c-9833-b06c2afe8b8b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:51.711476 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f6cac6-845b-476c-9833-b06c2afe8b8b service placement] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:51.738469 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 870/1731] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:40:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:40:51.741155 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f6cac6-845b-476c-9833-b06c2afe8b8b service placement] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:51.790000 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f6cac6-845b-476c-9833-b06c2afe8b8b service placement] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:51.806968 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 862/1732] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:40:51 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 522 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:40:51.820495 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031ffe21-9965-4183-97e1-b612ffdbdc4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:51.821266 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031ffe21-9965-4183-97e1-b612ffdbdc4f admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:51.821517 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031ffe21-9965-4183-97e1-b612ffdbdc4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:51.821800 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031ffe21-9965-4183-97e1-b612ffdbdc4f admin admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:51.823301 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcab8bc-cb4b-441d-af78-a2d276d561a7 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:51.826042 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031ffe21-9965-4183-97e1-b612ffdbdc4f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:40:51.826390 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031ffe21-9965-4183-97e1-b612ffdbdc4f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:51.828609 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031ffe21-9965-4183-97e1-b612ffdbdc4f admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:51.833623 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-8bcab8bc-cb4b-441d-af78-a2d276d561a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:51.906079 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 871/1733] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 24 06:40:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:40:51.906079 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031ffe21-9965-4183-97e1-b612ffdbdc4f admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:51.957735 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031ffe21-9965-4183-97e1-b612ffdbdc4f admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:51.983729 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb015244-db5c-4408-b3f8-0c40bcf1fb08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:51.983729 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb015244-db5c-4408-b3f8-0c40bcf1fb08 admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:51.983729 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb015244-db5c-4408-b3f8-0c40bcf1fb08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:51.983729 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb015244-db5c-4408-b3f8-0c40bcf1fb08 admin admin] PATH_INFO: `/v3/projects` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:51.984168 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb015244-db5c-4408-b3f8-0c40bcf1fb08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:51.987113 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb015244-db5c-4408-b3f8-0c40bcf1fb08 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:52.022586 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcab8bc-cb4b-441d-af78-a2d276d561a7 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:52.037081 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2c3720-060d-4f77-8442-21f68f6effcf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36639dce4f97412f89d71367f650b901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:52.040241 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2c3720-060d-4f77-8442-21f68f6effcf service cinder] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:52.040636 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2c3720-060d-4f77-8442-21f68f6effcf service cinder] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:52.040791 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2c3720-060d-4f77-8442-21f68f6effcf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:52.069870 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2c3720-060d-4f77-8442-21f68f6effcf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:52.072029 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2c3720-060d-4f77-8442-21f68f6effcf service cinder] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:52.116774 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 863/1734] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:40:51 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:40:52.138130 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2c3720-060d-4f77-8442-21f68f6effcf service cinder] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:52.166636 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 872/1735] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:40:51 2022] POST /identity/v3/projects => generated 363 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:40:52.189094 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2c3720-060d-4f77-8442-21f68f6effcf service cinder] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:52.401536 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52dcfef-6adc-431f-b9b2-f112b556a134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:52.408800 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52dcfef-6adc-431f-b9b2-f112b556a134 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:52.410440 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52dcfef-6adc-431f-b9b2-f112b556a134 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:52.410440 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52dcfef-6adc-431f-b9b2-f112b556a134 admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:52.410440 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52dcfef-6adc-431f-b9b2-f112b556a134 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:52.412244 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52dcfef-6adc-431f-b9b2-f112b556a134 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:52.591977 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 864/1736] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:40:52 2022] POST /identity/v3/users => generated 282 bytes in 455 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:40:52.596910 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb015244-db5c-4408-b3f8-0c40bcf1fb08 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:52.615369 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb015244-db5c-4408-b3f8-0c40bcf1fb08 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:52.629792 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4547e9-d675-4b9a-b7b1-c19bdb3249d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:52.630485 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4547e9-d675-4b9a-b7b1-c19bdb3249d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:52.630837 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4547e9-d675-4b9a-b7b1-c19bdb3249d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:52.631182 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4547e9-d675-4b9a-b7b1-c19bdb3249d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:52.632963 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4547e9-d675-4b9a-b7b1-c19bdb3249d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:52.636031 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4547e9-d675-4b9a-b7b1-c19bdb3249d6 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:52.676524 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 873/1737] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:40:52 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:40:52.687104 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52dcfef-6adc-431f-b9b2-f112b556a134 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:52.700748 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52dcfef-6adc-431f-b9b2-f112b556a134 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:52.726620 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5120506c-e992-4063-8dbb-b1c1590f9988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:52.727269 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5120506c-e992-4063-8dbb-b1c1590f9988 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:52.727554 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5120506c-e992-4063-8dbb-b1c1590f9988 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:52.727828 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5120506c-e992-4063-8dbb-b1c1590f9988 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:52.728741 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5120506c-e992-4063-8dbb-b1c1590f9988 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:52.735354 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5120506c-e992-4063-8dbb-b1c1590f9988 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:52.755619 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 865/1738] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:40:52 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:40:52.759521 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4547e9-d675-4b9a-b7b1-c19bdb3249d6 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:52.776850 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4547e9-d675-4b9a-b7b1-c19bdb3249d6 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:52.786887 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8f980a-409b-4e05-9f18-941dbb410957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:52.787954 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8f980a-409b-4e05-9f18-941dbb410957 admin admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:52.788504 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8f980a-409b-4e05-9f18-941dbb410957 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:52.789525 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8f980a-409b-4e05-9f18-941dbb410957 admin admin] PATH_INFO: `/v3/projects/7158a44909f1494b9f4259b74fc786f9/users/07828c08728d474cb4f4c6b21b90ebfb/roles/e1f9b167c51c4a388cbc9594db8a465d` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:52.828713 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8f980a-409b-4e05-9f18-941dbb410957 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7158a44909f1494b9f4259b74fc786f9, user_id=07828c08728d474cb4f4c6b21b90ebfb, role_id=e1f9b167c51c4a388cbc9594db8a465d)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:52.832484 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8f980a-409b-4e05-9f18-941dbb410957 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:52.950748 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 874/1739] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:40:52 2022] PUT /identity/v3/projects/7158a44909f1494b9f4259b74fc786f9/users/07828c08728d474cb4f4c6b21b90ebfb/roles/e1f9b167c51c4a388cbc9594db8a465d => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:40:52.971941 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333cdebd-9506-431b-ad94-658cb76edb2f None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:52.972293 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333cdebd-9506-431b-ad94-658cb76edb2f None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:52.972607 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333cdebd-9506-431b-ad94-658cb76edb2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:53.035669 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ee0dad-d5d0-4e99-b43e-21b9983f4e90 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:53.035933 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ee0dad-d5d0-4e99-b43e-21b9983f4e90 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:53.036344 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ee0dad-d5d0-4e99-b43e-21b9983f4e90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:53.090620 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-333cdebd-9506-431b-ad94-658cb76edb2f None None] MFA Rules not processed for user `07828c08728d474cb4f4c6b21b90ebfb`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:40:53.091871 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-333cdebd-9506-431b-ad94-658cb76edb2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:53.108079 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-71ee0dad-d5d0-4e99-b43e-21b9983f4e90 None None] MFA Rules not processed for user `8b9b29b53a7b490da29fbf6e5451c362`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:40:53.110418 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-71ee0dad-d5d0-4e99-b43e-21b9983f4e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:53.294673 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 866/1740] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:40:52 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:40:53.301259 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333cdebd-9506-431b-ad94-658cb76edb2f None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:53.349312 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 875/1741] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:40:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:40:53.507491 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333cdebd-9506-431b-ad94-658cb76edb2f None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:53.530168 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb7bf8d-0ebd-4249-884e-5e5481958d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:53.531276 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb7bf8d-0ebd-4249-884e-5e5481958d9d admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:53.533160 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb7bf8d-0ebd-4249-884e-5e5481958d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:53.533752 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb7bf8d-0ebd-4249-884e-5e5481958d9d admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:53.536114 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb7bf8d-0ebd-4249-884e-5e5481958d9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:53.542694 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb7bf8d-0ebd-4249-884e-5e5481958d9d admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:53.586169 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:40:53.715373 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:40:53.724359 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 867/1742] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:40:53 2022] POST /identity/v3/projects => generated 363 bytes in 424 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:40:53.730448 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ee0dad-d5d0-4e99-b43e-21b9983f4e90 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:53.747262 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ee0dad-d5d0-4e99-b43e-21b9983f4e90 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:53.922918 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0099300-3694-42aa-a471-4f8109c1531b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:53.924324 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0099300-3694-42aa-a471-4f8109c1531b admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:53.924798 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0099300-3694-42aa-a471-4f8109c1531b admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:53.925510 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0099300-3694-42aa-a471-4f8109c1531b admin admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:53.927113 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0099300-3694-42aa-a471-4f8109c1531b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:53.932122 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0099300-3694-42aa-a471-4f8109c1531b admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:54.101354 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 876/1743] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:40:53 2022] POST /identity/v3/users => generated 288 bytes in 372 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:40:54.106331 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb7bf8d-0ebd-4249-884e-5e5481958d9d admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:54.131626 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb7bf8d-0ebd-4249-884e-5e5481958d9d admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:54.183859 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615cb572-7a7b-4708-af49-943a8bd1f904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:54.186747 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615cb572-7a7b-4708-af49-943a8bd1f904 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:54.193199 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615cb572-7a7b-4708-af49-943a8bd1f904 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:54.193915 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615cb572-7a7b-4708-af49-943a8bd1f904 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:54.202237 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615cb572-7a7b-4708-af49-943a8bd1f904 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:54.210453 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0099300-3694-42aa-a471-4f8109c1531b admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:54.217942 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615cb572-7a7b-4708-af49-943a8bd1f904 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:54.315200 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 868/1744] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:40:54 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:40:54.323766 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615cb572-7a7b-4708-af49-943a8bd1f904 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:54.388348 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615cb572-7a7b-4708-af49-943a8bd1f904 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:54.419928 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a1b4ac-6337-45b3-940b-4c3ec3efbefc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:54.420635 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a1b4ac-6337-45b3-940b-4c3ec3efbefc admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:54.424712 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a1b4ac-6337-45b3-940b-4c3ec3efbefc admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:54.425026 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a1b4ac-6337-45b3-940b-4c3ec3efbefc admin admin] PATH_INFO: `/v3/projects/0a27f1fc29fc4e0eb953a5aee22f5538/users/27503206d9b84c69a74457fa777a17fc/roles/486144ad1bcd417e86acdbc1d14327bf` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:54.538185 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a1b4ac-6337-45b3-940b-4c3ec3efbefc admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a27f1fc29fc4e0eb953a5aee22f5538, user_id=27503206d9b84c69a74457fa777a17fc, role_id=486144ad1bcd417e86acdbc1d14327bf)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:54.540903 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a1b4ac-6337-45b3-940b-4c3ec3efbefc admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:54.606674 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0099300-3694-42aa-a471-4f8109c1531b admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:54.635274 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c752a19c-da37-4756-b575-d472734d91df tempest-VolumeQuotaClassesTest-486985756 tempest-VolumeQuotaClassesTest-486985756-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a224fd5432946e3be47435a42434bd6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa545d11e3f04f2488f53e6f7d5873b8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:54.642903 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c752a19c-da37-4756-b575-d472734d91df tempest-VolumeQuotaClassesTest-486985756 tempest-VolumeQuotaClassesTest-486985756-project-admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:54.642903 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c752a19c-da37-4756-b575-d472734d91df tempest-VolumeQuotaClassesTest-486985756 tempest-VolumeQuotaClassesTest-486985756-project-admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:54.642903 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c752a19c-da37-4756-b575-d472734d91df tempest-VolumeQuotaClassesTest-486985756 tempest-VolumeQuotaClassesTest-486985756-project-admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:54.642903 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c752a19c-da37-4756-b575-d472734d91df tempest-VolumeQuotaClassesTest-486985756 tempest-VolumeQuotaClassesTest-486985756-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:40:54.642903 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c752a19c-da37-4756-b575-d472734d91df tempest-VolumeQuotaClassesTest-486985756 tempest-VolumeQuotaClassesTest-486985756-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:54.644593 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c752a19c-da37-4756-b575-d472734d91df tempest-VolumeQuotaClassesTest-486985756 tempest-VolumeQuotaClassesTest-486985756-project-admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:54.678907 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 869/1745] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:40:54 2022] PUT /identity/v3/projects/0a27f1fc29fc4e0eb953a5aee22f5538/users/27503206d9b84c69a74457fa777a17fc/roles/486144ad1bcd417e86acdbc1d14327bf => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:40:54.680432 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b4ab5-64c0-4ef3-84da-dcd67f412eb9 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:54.682352 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b4ab5-64c0-4ef3-84da-dcd67f412eb9 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:54.688480 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b4ab5-64c0-4ef3-84da-dcd67f412eb9 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:54.698146 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 870/1746] 10.222.0.47 () {54 vars in 912 bytes} [Wed Aug 24 06:40:54 2022] GET /identity => generated 270 bytes in 19 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:40:54.700810 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822b4ab5-64c0-4ef3-84da-dcd67f412eb9 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:54.725984 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 877/1747] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 24 06:40:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 517 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:40:54.727317 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c4fec0-38e1-4caf-a4cb-62aaa3423369 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:54.727608 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c4fec0-38e1-4caf-a4cb-62aaa3423369 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:54.728143 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c4fec0-38e1-4caf-a4cb-62aaa3423369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:54.815681 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-f0c4fec0-38e1-4caf-a4cb-62aaa3423369 None None] MFA Rules not processed for user `f0ba3f03ec15475ab6ebf64994ad742c`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:40:54.819132 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-f0c4fec0-38e1-4caf-a4cb-62aaa3423369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:54.945335 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822b4ab5-64c0-4ef3-84da-dcd67f412eb9 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:54.951475 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a34de9-b61c-4758-85d1-2dcc2b6258e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:54.952320 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a34de9-b61c-4758-85d1-2dcc2b6258e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:54.953307 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a34de9-b61c-4758-85d1-2dcc2b6258e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:54.954187 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a34de9-b61c-4758-85d1-2dcc2b6258e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:54.955024 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 878/1748] 10.222.0.47 () {58 vars in 1009 bytes} [Wed Aug 24 06:40:54 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:40:54.956034 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c4fec0-38e1-4caf-a4cb-62aaa3423369 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:54.962037 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a34de9-b61c-4758-85d1-2dcc2b6258e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:54.964080 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a34de9-b61c-4758-85d1-2dcc2b6258e2 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:54.977070 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 871/1749] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:40:54 2022] GET /identity/v3/roles => generated 1420 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:40:54.981222 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a34de9-b61c-4758-85d1-2dcc2b6258e2 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:55.021586 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a34de9-b61c-4758-85d1-2dcc2b6258e2 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:55.028831 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66da02e4-768b-4407-8536-466f47def8a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:55.029929 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66da02e4-768b-4407-8536-466f47def8a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:55.030423 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66da02e4-768b-4407-8536-466f47def8a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:55.031949 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66da02e4-768b-4407-8536-466f47def8a1 admin admin] PATH_INFO: `/v3/projects/0a27f1fc29fc4e0eb953a5aee22f5538/users/27503206d9b84c69a74457fa777a17fc/roles/486144ad1bcd417e86acdbc1d14327bf` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:55.035897 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66da02e4-768b-4407-8536-466f47def8a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a27f1fc29fc4e0eb953a5aee22f5538, user_id=27503206d9b84c69a74457fa777a17fc, role_id=486144ad1bcd417e86acdbc1d14327bf)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:55.039214 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66da02e4-768b-4407-8536-466f47def8a1 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:55.062538 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 872/1750] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:40:54 2022] PUT /identity/v3/projects/0a27f1fc29fc4e0eb953a5aee22f5538/users/27503206d9b84c69a74457fa777a17fc/roles/486144ad1bcd417e86acdbc1d14327bf => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:40:55.064489 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66da02e4-768b-4407-8536-466f47def8a1 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:55.071587 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-66da02e4-768b-4407-8536-466f47def8a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:55.166012 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c4fec0-38e1-4caf-a4cb-62aaa3423369 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:55.247265 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66da02e4-768b-4407-8536-466f47def8a1 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:55.251863 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c28bd1b-df20-4489-8249-5d00df1ca409 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:55.252728 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c28bd1b-df20-4489-8249-5d00df1ca409 service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:55.253014 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c28bd1b-df20-4489-8249-5d00df1ca409 service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:55.253324 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c28bd1b-df20-4489-8249-5d00df1ca409 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:55.263339 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c28bd1b-df20-4489-8249-5d00df1ca409 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:55.265053 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c28bd1b-df20-4489-8249-5d00df1ca409 service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:55.275678 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 873/1751] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:40:55 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:40:55.285945 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9b8241-536e-4128-a18c-ee45afe23bae tempest-VolumeQuotaClassesTest-486985756 tempest-VolumeQuotaClassesTest-486985756-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a224fd5432946e3be47435a42434bd6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa545d11e3f04f2488f53e6f7d5873b8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:55.290579 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200bbd89-6da0-4a7d-aae9-3aef4c550295 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:55.290700 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9b8241-536e-4128-a18c-ee45afe23bae tempest-VolumeQuotaClassesTest-486985756 tempest-VolumeQuotaClassesTest-486985756-project-admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:55.290793 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200bbd89-6da0-4a7d-aae9-3aef4c550295 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:55.290865 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9b8241-536e-4128-a18c-ee45afe23bae tempest-VolumeQuotaClassesTest-486985756 tempest-VolumeQuotaClassesTest-486985756-project-admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:55.290937 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200bbd89-6da0-4a7d-aae9-3aef4c550295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:55.291004 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9b8241-536e-4128-a18c-ee45afe23bae tempest-VolumeQuotaClassesTest-486985756 tempest-VolumeQuotaClassesTest-486985756-project-admin] PATH_INFO: `/v3/projects` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:55.291004 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9b8241-536e-4128-a18c-ee45afe23bae tempest-VolumeQuotaClassesTest-486985756 tempest-VolumeQuotaClassesTest-486985756-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:55.291004 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9b8241-536e-4128-a18c-ee45afe23bae tempest-VolumeQuotaClassesTest-486985756 tempest-VolumeQuotaClassesTest-486985756-project-admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:55.377773 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-200bbd89-6da0-4a7d-aae9-3aef4c550295 None None] MFA Rules not processed for user `27503206d9b84c69a74457fa777a17fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:40:55.382321 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-200bbd89-6da0-4a7d-aae9-3aef4c550295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:55.522719 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 879/1752] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:40:54 2022] POST /identity/v3/projects => generated 354 bytes in 565 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:40:55.526880 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9b8241-536e-4128-a18c-ee45afe23bae tempest-VolumeQuotaClassesTest-486985756 tempest-VolumeQuotaClassesTest-486985756-project-admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:55.651645 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 874/1753] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:40:55 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 362 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:40:55.673774 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200bbd89-6da0-4a7d-aae9-3aef4c550295 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:55.679077 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-200bbd89-6da0-4a7d-aae9-3aef4c550295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:55.729740 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9b8241-536e-4128-a18c-ee45afe23bae tempest-VolumeQuotaClassesTest-486985756 tempest-VolumeQuotaClassesTest-486985756-project-admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:55.750127 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323b7e3d-b48f-479c-9b31-d20ce98b515c tempest-VolumeQuotaClassesTest-486985756 tempest-VolumeQuotaClassesTest-486985756-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a224fd5432946e3be47435a42434bd6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa545d11e3f04f2488f53e6f7d5873b8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:55.751205 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323b7e3d-b48f-479c-9b31-d20ce98b515c tempest-VolumeQuotaClassesTest-486985756 tempest-VolumeQuotaClassesTest-486985756-project-admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:55.751556 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323b7e3d-b48f-479c-9b31-d20ce98b515c tempest-VolumeQuotaClassesTest-486985756 tempest-VolumeQuotaClassesTest-486985756-project-admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:55.751920 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323b7e3d-b48f-479c-9b31-d20ce98b515c tempest-VolumeQuotaClassesTest-486985756 tempest-VolumeQuotaClassesTest-486985756-project-admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:55.752688 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323b7e3d-b48f-479c-9b31-d20ce98b515c tempest-VolumeQuotaClassesTest-486985756 tempest-VolumeQuotaClassesTest-486985756-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:40:55.753065 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323b7e3d-b48f-479c-9b31-d20ce98b515c tempest-VolumeQuotaClassesTest-486985756 tempest-VolumeQuotaClassesTest-486985756-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:55.754921 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323b7e3d-b48f-479c-9b31-d20ce98b515c tempest-VolumeQuotaClassesTest-486985756 tempest-VolumeQuotaClassesTest-486985756-project-admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:55.807364 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 880/1754] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 24 06:40:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:40:55.879867 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200bbd89-6da0-4a7d-aae9-3aef4c550295 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:55.907468 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323b7e3d-b48f-479c-9b31-d20ce98b515c tempest-VolumeQuotaClassesTest-486985756 tempest-VolumeQuotaClassesTest-486985756-project-admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:55.927524 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323b7e3d-b48f-479c-9b31-d20ce98b515c tempest-VolumeQuotaClassesTest-486985756 tempest-VolumeQuotaClassesTest-486985756-project-admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:55.943445 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716dd999-ce99-4a8c-b254-ba3baf4a7216 tempest-VolumeQuotaClassesTest-486985756 tempest-VolumeQuotaClassesTest-486985756-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a224fd5432946e3be47435a42434bd6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fa545d11e3f04f2488f53e6f7d5873b8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:55.946347 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716dd999-ce99-4a8c-b254-ba3baf4a7216 tempest-VolumeQuotaClassesTest-486985756 tempest-VolumeQuotaClassesTest-486985756-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:55.946347 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716dd999-ce99-4a8c-b254-ba3baf4a7216 tempest-VolumeQuotaClassesTest-486985756 tempest-VolumeQuotaClassesTest-486985756-project-admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:55.946347 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716dd999-ce99-4a8c-b254-ba3baf4a7216 tempest-VolumeQuotaClassesTest-486985756 tempest-VolumeQuotaClassesTest-486985756-project-admin] PATH_INFO: `/v3/projects/2c7210a5b2e944308d9e6dbd40c34aa6` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:55.952845 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716dd999-ce99-4a8c-b254-ba3baf4a7216 tempest-VolumeQuotaClassesTest-486985756 tempest-VolumeQuotaClassesTest-486985756-project-admin] RBAC: Authorizing `identity:delete_project(project_id=2c7210a5b2e944308d9e6dbd40c34aa6)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:55.962527 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716dd999-ce99-4a8c-b254-ba3baf4a7216 tempest-VolumeQuotaClassesTest-486985756 tempest-VolumeQuotaClassesTest-486985756-project-admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:56.026940 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a5789e-5e1c-4a17-83fb-ee4dbca39522 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36639dce4f97412f89d71367f650b901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:56.027584 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a5789e-5e1c-4a17-83fb-ee4dbca39522 service cinder] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:56.027870 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a5789e-5e1c-4a17-83fb-ee4dbca39522 service cinder] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:56.028147 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a5789e-5e1c-4a17-83fb-ee4dbca39522 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:56.176064 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:40:56.210395 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-716dd999-ce99-4a8c-b254-ba3baf4a7216 tempest-VolumeQuotaClassesTest-486985756 tempest-VolumeQuotaClassesTest-486985756-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c7210a5b2e944308d9e6dbd40c34aa6', 'request_id': 'req-716dd999-ce99-4a8c-b254-ba3baf4a7216'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:40:56.292069 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a5789e-5e1c-4a17-83fb-ee4dbca39522 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:56.294110 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a5789e-5e1c-4a17-83fb-ee4dbca39522 service cinder] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:56.326547 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 875/1755] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:40:55 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 656 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:40:56.326887 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a5789e-5e1c-4a17-83fb-ee4dbca39522 service cinder] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:56.333044 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-93a5789e-5e1c-4a17-83fb-ee4dbca39522 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:56.407957 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-716dd999-ce99-4a8c-b254-ba3baf4a7216 tempest-VolumeQuotaClassesTest-486985756 tempest-VolumeQuotaClassesTest-486985756-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c7210a5b2e944308d9e6dbd40c34aa6', 'request_id': 'req-716dd999-ce99-4a8c-b254-ba3baf4a7216'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:40:56.422384 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-716dd999-ce99-4a8c-b254-ba3baf4a7216 tempest-VolumeQuotaClassesTest-486985756 tempest-VolumeQuotaClassesTest-486985756-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c7210a5b2e944308d9e6dbd40c34aa6', 'request_id': 'req-716dd999-ce99-4a8c-b254-ba3baf4a7216'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:40:56.514554 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-716dd999-ce99-4a8c-b254-ba3baf4a7216 tempest-VolumeQuotaClassesTest-486985756 tempest-VolumeQuotaClassesTest-486985756-project-admin] The token cache is being invalidate because project 2c7210a5b2e944308d9e6dbd40c34aa6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81723) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:40:56.518736 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-716dd999-ce99-4a8c-b254-ba3baf4a7216 tempest-VolumeQuotaClassesTest-486985756 tempest-VolumeQuotaClassesTest-486985756-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:40:56.524248 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-716dd999-ce99-4a8c-b254-ba3baf4a7216 tempest-VolumeQuotaClassesTest-486985756 tempest-VolumeQuotaClassesTest-486985756-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:40:56.535999 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 881/1756] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:40:55 2022] DELETE /identity/v3/projects/2c7210a5b2e944308d9e6dbd40c34aa6 => generated 0 bytes in 630 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:40:56.558972 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a5789e-5e1c-4a17-83fb-ee4dbca39522 service cinder] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:56.641254 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24cca9a-ca01-4311-9662-a5a798756f51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:56.642205 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24cca9a-ca01-4311-9662-a5a798756f51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:56.642640 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24cca9a-ca01-4311-9662-a5a798756f51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:56.643073 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24cca9a-ca01-4311-9662-a5a798756f51 admin admin] PATH_INFO: `/v3/users/bbbd204d171e415ebd1b928e00d3303f` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:56.645787 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24cca9a-ca01-4311-9662-a5a798756f51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbbd204d171e415ebd1b928e00d3303f)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:56.647962 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24cca9a-ca01-4311-9662-a5a798756f51 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:56.753925 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716dd999-ce99-4a8c-b254-ba3baf4a7216 tempest-VolumeQuotaClassesTest-486985756 tempest-VolumeQuotaClassesTest-486985756-project-admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:56.757756 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-716dd999-ce99-4a8c-b254-ba3baf4a7216 tempest-VolumeQuotaClassesTest-486985756 tempest-VolumeQuotaClassesTest-486985756-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:56.851481 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716dd999-ce99-4a8c-b254-ba3baf4a7216 tempest-VolumeQuotaClassesTest-486985756 tempest-VolumeQuotaClassesTest-486985756-project-admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:56.856924 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8809ca5-f09c-40b1-b2b7-e1f600fa028d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36639dce4f97412f89d71367f650b901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:56.858040 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8809ca5-f09c-40b1-b2b7-e1f600fa028d service cinder] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:56.858333 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8809ca5-f09c-40b1-b2b7-e1f600fa028d service cinder] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:56.858761 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8809ca5-f09c-40b1-b2b7-e1f600fa028d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:56.861097 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-f8809ca5-f09c-40b1-b2b7-e1f600fa028d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:56.975208 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-b24cca9a-ca01-4311-9662-a5a798756f51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbbd204d171e415ebd1b928e00d3303f'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:40:56.976251 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-b24cca9a-ca01-4311-9662-a5a798756f51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbbd204d171e415ebd1b928e00d3303f'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:40:56.977231 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-b24cca9a-ca01-4311-9662-a5a798756f51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbbd204d171e415ebd1b928e00d3303f'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:40:57.018260 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-b24cca9a-ca01-4311-9662-a5a798756f51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbbd204d171e415ebd1b928e00d3303f'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:40:57.018452 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8809ca5-f09c-40b1-b2b7-e1f600fa028d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:57.019066 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8809ca5-f09c-40b1-b2b7-e1f600fa028d service cinder] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:57.052706 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 882/1757] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:40:56 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:40:57.101941 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49932caa-ac72-4e5e-8355-8bb76d8c4469 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:57.102256 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49932caa-ac72-4e5e-8355-8bb76d8c4469 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:57.102542 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49932caa-ac72-4e5e-8355-8bb76d8c4469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:57.154659 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-49932caa-ac72-4e5e-8355-8bb76d8c4469 None None] MFA Rules not processed for user `02ad6cdc4ffe4917a7bbf6814297f980`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:40:57.159636 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-49932caa-ac72-4e5e-8355-8bb76d8c4469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:57.236116 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-b24cca9a-ca01-4311-9662-a5a798756f51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbbd204d171e415ebd1b928e00d3303f'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:40:57.285684 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 876/1758] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:40:56 2022] DELETE /identity/v3/users/bbbd204d171e415ebd1b928e00d3303f => generated 0 bytes in 956 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:40:57.287529 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d545b6-cd8b-44fe-88d9-10b25b7b2d53 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:57.287878 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d545b6-cd8b-44fe-88d9-10b25b7b2d53 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:57.288169 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d545b6-cd8b-44fe-88d9-10b25b7b2d53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:57.337078 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-73d545b6-cd8b-44fe-88d9-10b25b7b2d53 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:40:57.338902 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-73d545b6-cd8b-44fe-88d9-10b25b7b2d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:57.340431 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:40:57.347736 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 883/1759] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:40:57 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:40:57.490713 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 877/1760] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:40:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:40:57.504200 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49932caa-ac72-4e5e-8355-8bb76d8c4469 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:57.506802 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-49932caa-ac72-4e5e-8355-8bb76d8c4469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:57.724145 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49932caa-ac72-4e5e-8355-8bb76d8c4469 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:57.946093 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd3adf6-29ce-4f82-9d04-551f1cc8d49e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:57.947192 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd3adf6-29ce-4f82-9d04-551f1cc8d49e service neutron] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:57.947586 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd3adf6-29ce-4f82-9d04-551f1cc8d49e service neutron] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:57.948131 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd3adf6-29ce-4f82-9d04-551f1cc8d49e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:57.960302 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd3adf6-29ce-4f82-9d04-551f1cc8d49e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:57.961907 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd3adf6-29ce-4f82-9d04-551f1cc8d49e service neutron] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:57.996796 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 884/1761] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:40:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:40:58.108362 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d545b6-cd8b-44fe-88d9-10b25b7b2d53 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:58.111632 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-73d545b6-cd8b-44fe-88d9-10b25b7b2d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:58.174751 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d545b6-cd8b-44fe-88d9-10b25b7b2d53 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:58.220356 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0579f06-7ca1-44e7-9b85-6db8d8e4847c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:58.228596 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0579f06-7ca1-44e7-9b85-6db8d8e4847c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:58.229311 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0579f06-7ca1-44e7-9b85-6db8d8e4847c admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:58.230757 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0579f06-7ca1-44e7-9b85-6db8d8e4847c admin admin] PATH_INFO: `/v3/users/dc2801778ce74ada8dc8aa4f51526920` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:58.232736 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0579f06-7ca1-44e7-9b85-6db8d8e4847c admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc2801778ce74ada8dc8aa4f51526920)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:58.236175 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0579f06-7ca1-44e7-9b85-6db8d8e4847c admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:58.541488 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d150c95-9e94-4786-86ce-21dced5e87d9 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:58.554488 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d150c95-9e94-4786-86ce-21dced5e87d9 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:58.554488 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d150c95-9e94-4786-86ce-21dced5e87d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:58.565893 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-6d150c95-9e94-4786-86ce-21dced5e87d9 None None] MFA Rules not processed for user `52f796d5013d4a31b107fb86c5523c9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:40:58.567253 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-6d150c95-9e94-4786-86ce-21dced5e87d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:58.680643 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-c0579f06-7ca1-44e7-9b85-6db8d8e4847c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc2801778ce74ada8dc8aa4f51526920'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:40:58.681953 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-c0579f06-7ca1-44e7-9b85-6db8d8e4847c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc2801778ce74ada8dc8aa4f51526920'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:40:58.683698 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-c0579f06-7ca1-44e7-9b85-6db8d8e4847c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc2801778ce74ada8dc8aa4f51526920'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:40:58.713243 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 885/1762] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:40:58 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:40:58.763419 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-c0579f06-7ca1-44e7-9b85-6db8d8e4847c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc2801778ce74ada8dc8aa4f51526920'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:40:58.919841 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-c0579f06-7ca1-44e7-9b85-6db8d8e4847c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc2801778ce74ada8dc8aa4f51526920'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:40:58.947245 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601d6c61-bdc3-4afe-9178-51eeb0493138 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:58.947785 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601d6c61-bdc3-4afe-9178-51eeb0493138 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:58.947785 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601d6c61-bdc3-4afe-9178-51eeb0493138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:58.968944 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 878/1763] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:40:58 2022] DELETE /identity/v3/users/dc2801778ce74ada8dc8aa4f51526920 => generated 0 bytes in 864 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:40:58.975593 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c608dbe5-ab40-4e56-ade8-a69a6b8102d2 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:58.976169 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c608dbe5-ab40-4e56-ade8-a69a6b8102d2 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:58.976663 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c608dbe5-ab40-4e56-ade8-a69a6b8102d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:59.015617 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-c608dbe5-ab40-4e56-ade8-a69a6b8102d2 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:40:59.019287 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-c608dbe5-ab40-4e56-ade8-a69a6b8102d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:59.079572 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-601d6c61-bdc3-4afe-9178-51eeb0493138 None None] MFA Rules not processed for user `ea70acd744994250a2905b5dd7163dfb`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:40:59.081102 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-601d6c61-bdc3-4afe-9178-51eeb0493138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:59.147305 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 879/1764] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:40:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:40:59.157620 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c608dbe5-ab40-4e56-ade8-a69a6b8102d2 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:59.160176 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-c608dbe5-ab40-4e56-ade8-a69a6b8102d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:59.311690 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 886/1765] 10.222.0.47 () {58 vars in 1009 bytes} [Wed Aug 24 06:40:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 365 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Aug 24 06:40:59.312924 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601d6c61-bdc3-4afe-9178-51eeb0493138 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:59.318154 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-601d6c61-bdc3-4afe-9178-51eeb0493138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:40:59.359528 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601d6c61-bdc3-4afe-9178-51eeb0493138 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:59.378364 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ac396e-6f34-4fa1-9c13-e3294d5144dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:59.378978 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ac396e-6f34-4fa1-9c13-e3294d5144dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:59.379264 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ac396e-6f34-4fa1-9c13-e3294d5144dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:59.379698 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ac396e-6f34-4fa1-9c13-e3294d5144dc admin admin] PATH_INFO: `/v3/projects/324e3f0a469f48f3943ecfc9ee9c5749` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:59.389168 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ac396e-6f34-4fa1-9c13-e3294d5144dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=324e3f0a469f48f3943ecfc9ee9c5749)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:59.395033 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ac396e-6f34-4fa1-9c13-e3294d5144dc admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:59.459094 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c608dbe5-ab40-4e56-ade8-a69a6b8102d2 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:59.481301 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f8b9c4-9e15-49c3-b8ca-99c92421dea7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:40:59.482223 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f8b9c4-9e15-49c3-b8ca-99c92421dea7 service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:40:59.486272 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f8b9c4-9e15-49c3-b8ca-99c92421dea7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:40:59.490656 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f8b9c4-9e15-49c3-b8ca-99c92421dea7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:40:59.525782 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f8b9c4-9e15-49c3-b8ca-99c92421dea7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:40:59.529623 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f8b9c4-9e15-49c3-b8ca-99c92421dea7 service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:40:59.594753 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 880/1766] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:40:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:40:59.703523 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-57ac396e-6f34-4fa1-9c13-e3294d5144dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '324e3f0a469f48f3943ecfc9ee9c5749', 'request_id': 'req-57ac396e-6f34-4fa1-9c13-e3294d5144dc'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:40:59.835940 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-57ac396e-6f34-4fa1-9c13-e3294d5144dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '324e3f0a469f48f3943ecfc9ee9c5749', 'request_id': 'req-57ac396e-6f34-4fa1-9c13-e3294d5144dc'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:40:59.848112 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-57ac396e-6f34-4fa1-9c13-e3294d5144dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '324e3f0a469f48f3943ecfc9ee9c5749', 'request_id': 'req-57ac396e-6f34-4fa1-9c13-e3294d5144dc'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:40:59.867315 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-57ac396e-6f34-4fa1-9c13-e3294d5144dc admin admin] The token cache is being invalidate because project 324e3f0a469f48f3943ecfc9ee9c5749 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81723) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:40:59.867865 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-57ac396e-6f34-4fa1-9c13-e3294d5144dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:40:59.868801 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-57ac396e-6f34-4fa1-9c13-e3294d5144dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:40:59.872345 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 887/1767] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:40:59 2022] DELETE /identity/v3/projects/324e3f0a469f48f3943ecfc9ee9c5749 => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:40:59.879843 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f8b9c4-9e15-49c3-b8ca-99c92421dea7 service neutron] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:59.906092 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f8b9c4-9e15-49c3-b8ca-99c92421dea7 service neutron] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:40:59.948966 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:40:59.962030 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ac396e-6f34-4fa1-9c13-e3294d5144dc admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:40:59.969138 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-57ac396e-6f34-4fa1-9c13-e3294d5144dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:00.047393 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:41:00.119489 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75787a36-db21-4492-bb8d-539267a421cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:00.119489 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75787a36-db21-4492-bb8d-539267a421cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:00.119489 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75787a36-db21-4492-bb8d-539267a421cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:00.119489 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75787a36-db21-4492-bb8d-539267a421cc admin admin] PATH_INFO: `/v3/users/1abd7b7f2cd54f1b8bfe68807041afe7` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:00.120161 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75787a36-db21-4492-bb8d-539267a421cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=1abd7b7f2cd54f1b8bfe68807041afe7)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:00.122153 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75787a36-db21-4492-bb8d-539267a421cc admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:00.273757 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ac396e-6f34-4fa1-9c13-e3294d5144dc admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:00.279408 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc36bcef-8ca5-49e0-a998-222fb5f837d7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f1cbaf2cd6427585f298c9d58c319a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:00.281260 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc36bcef-8ca5-49e0-a998-222fb5f837d7 service swift] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:00.281879 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc36bcef-8ca5-49e0-a998-222fb5f837d7 service swift] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:00.281992 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc36bcef-8ca5-49e0-a998-222fb5f837d7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:00.284520 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-fc36bcef-8ca5-49e0-a998-222fb5f837d7 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:00.411374 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc36bcef-8ca5-49e0-a998-222fb5f837d7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:00.416545 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc36bcef-8ca5-49e0-a998-222fb5f837d7 service swift] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:00.431363 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 888/1768] 10.222.0.47 () {60 vars in 1384 bytes} [Wed Aug 24 06:40:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 06:41:00.484159 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc36bcef-8ca5-49e0-a998-222fb5f837d7 service swift] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:00.496388 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-fc36bcef-8ca5-49e0-a998-222fb5f837d7 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:00.510074 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-75787a36-db21-4492-bb8d-539267a421cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1abd7b7f2cd54f1b8bfe68807041afe7'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:00.511445 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-75787a36-db21-4492-bb8d-539267a421cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1abd7b7f2cd54f1b8bfe68807041afe7'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:00.512974 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-75787a36-db21-4492-bb8d-539267a421cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1abd7b7f2cd54f1b8bfe68807041afe7'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:00.528326 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-75787a36-db21-4492-bb8d-539267a421cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1abd7b7f2cd54f1b8bfe68807041afe7'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:00.600718 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc36bcef-8ca5-49e0-a998-222fb5f837d7 service swift] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:00.617445 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5e934b-f522-49fc-a9cd-03eaf13e0fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:00.618258 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e934b-f522-49fc-a9cd-03eaf13e0fa9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:00.618774 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e934b-f522-49fc-a9cd-03eaf13e0fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:00.619184 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e934b-f522-49fc-a9cd-03eaf13e0fa9 admin admin] PATH_INFO: `/v3/projects/50e9d12bda7240edb650f24518264ef9` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:00.660765 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5e934b-f522-49fc-a9cd-03eaf13e0fa9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=50e9d12bda7240edb650f24518264ef9)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:00.666922 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-75787a36-db21-4492-bb8d-539267a421cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1abd7b7f2cd54f1b8bfe68807041afe7'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:00.676058 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5e934b-f522-49fc-a9cd-03eaf13e0fa9 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:00.751392 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 881/1769] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:40:59 2022] DELETE /identity/v3/users/1abd7b7f2cd54f1b8bfe68807041afe7 => generated 0 bytes in 873 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:41:01.206497 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-7c5e934b-f522-49fc-a9cd-03eaf13e0fa9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50e9d12bda7240edb650f24518264ef9', 'request_id': 'req-7c5e934b-f522-49fc-a9cd-03eaf13e0fa9'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:01.331033 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc663f28-e905-4b6b-a66a-32e6be2741f5 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:01.331479 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc663f28-e905-4b6b-a66a-32e6be2741f5 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:01.331808 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc663f28-e905-4b6b-a66a-32e6be2741f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:01.341606 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-7c5e934b-f522-49fc-a9cd-03eaf13e0fa9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50e9d12bda7240edb650f24518264ef9', 'request_id': 'req-7c5e934b-f522-49fc-a9cd-03eaf13e0fa9'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:01.359519 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-7c5e934b-f522-49fc-a9cd-03eaf13e0fa9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50e9d12bda7240edb650f24518264ef9', 'request_id': 'req-7c5e934b-f522-49fc-a9cd-03eaf13e0fa9'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:01.429080 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-dc663f28-e905-4b6b-a66a-32e6be2741f5 None None] MFA Rules not processed for user `f0ba3f03ec15475ab6ebf64994ad742c`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:41:01.435098 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-dc663f28-e905-4b6b-a66a-32e6be2741f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:01.448312 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-7c5e934b-f522-49fc-a9cd-03eaf13e0fa9 admin admin] The token cache is being invalidate because project 50e9d12bda7240edb650f24518264ef9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81723) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:41:01.448312 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-7c5e934b-f522-49fc-a9cd-03eaf13e0fa9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:01.450973 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-7c5e934b-f522-49fc-a9cd-03eaf13e0fa9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:01.460902 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 889/1770] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:41:00 2022] DELETE /identity/v3/projects/50e9d12bda7240edb650f24518264ef9 => generated 0 bytes in 978 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:41:01.465884 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5e934b-f522-49fc-a9cd-03eaf13e0fa9 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:01.471471 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-7c5e934b-f522-49fc-a9cd-03eaf13e0fa9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:01.687214 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5e934b-f522-49fc-a9cd-03eaf13e0fa9 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:01.697662 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dfd1dc-be3b-4d9e-bc87-0baa6918720e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:01.698454 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dfd1dc-be3b-4d9e-bc87-0baa6918720e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:01.699262 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dfd1dc-be3b-4d9e-bc87-0baa6918720e admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:01.699673 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dfd1dc-be3b-4d9e-bc87-0baa6918720e admin admin] PATH_INFO: `/v3/users/9a224fd5432946e3be47435a42434bd6` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:01.705579 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dfd1dc-be3b-4d9e-bc87-0baa6918720e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a224fd5432946e3be47435a42434bd6)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:01.707614 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dfd1dc-be3b-4d9e-bc87-0baa6918720e admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:01.761161 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 882/1771] 10.222.0.67 () {58 vars in 1015 bytes} [Wed Aug 24 06:41:01 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 435 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:41:01.791137 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc663f28-e905-4b6b-a66a-32e6be2741f5 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:01.793487 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-dc663f28-e905-4b6b-a66a-32e6be2741f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:01.830074 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc663f28-e905-4b6b-a66a-32e6be2741f5 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:01.853821 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245d6935-78ef-4c0f-8e88-d1c84da6cb0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:01.854742 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245d6935-78ef-4c0f-8e88-d1c84da6cb0b service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:01.854900 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245d6935-78ef-4c0f-8e88-d1c84da6cb0b service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:01.855246 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245d6935-78ef-4c0f-8e88-d1c84da6cb0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:01.884115 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245d6935-78ef-4c0f-8e88-d1c84da6cb0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:01.886113 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245d6935-78ef-4c0f-8e88-d1c84da6cb0b service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:01.900951 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 883/1772] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:41:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:41:02.112349 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-41dfd1dc-be3b-4d9e-bc87-0baa6918720e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a224fd5432946e3be47435a42434bd6'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:02.115213 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-41dfd1dc-be3b-4d9e-bc87-0baa6918720e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a224fd5432946e3be47435a42434bd6'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:02.125454 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-41dfd1dc-be3b-4d9e-bc87-0baa6918720e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a224fd5432946e3be47435a42434bd6'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:02.238735 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-41dfd1dc-be3b-4d9e-bc87-0baa6918720e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a224fd5432946e3be47435a42434bd6'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:02.278157 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-41dfd1dc-be3b-4d9e-bc87-0baa6918720e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a224fd5432946e3be47435a42434bd6'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:02.355170 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 890/1773] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:41:01 2022] DELETE /identity/v3/users/9a224fd5432946e3be47435a42434bd6 => generated 0 bytes in 894 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:41:02.796948 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:41:03.405591 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245d6935-78ef-4c0f-8e88-d1c84da6cb0b service neutron] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:03.410232 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-245d6935-78ef-4c0f-8e88-d1c84da6cb0b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:03.495873 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245d6935-78ef-4c0f-8e88-d1c84da6cb0b service neutron] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:03.681592 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8834a7d-9478-401e-97ca-9541e7cbb723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:03.682866 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8834a7d-9478-401e-97ca-9541e7cbb723 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:03.683474 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8834a7d-9478-401e-97ca-9541e7cbb723 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:03.683901 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8834a7d-9478-401e-97ca-9541e7cbb723 admin admin] PATH_INFO: `/v3/projects/33f9a38265ce4d65aa35b54e01a2a906` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:03.686020 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8834a7d-9478-401e-97ca-9541e7cbb723 admin admin] RBAC: Authorizing `identity:delete_project(project_id=33f9a38265ce4d65aa35b54e01a2a906)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:03.688605 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8834a7d-9478-401e-97ca-9541e7cbb723 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:03.959731 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-c8834a7d-9478-401e-97ca-9541e7cbb723 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33f9a38265ce4d65aa35b54e01a2a906', 'request_id': 'req-c8834a7d-9478-401e-97ca-9541e7cbb723'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:04.001543 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-c8834a7d-9478-401e-97ca-9541e7cbb723 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33f9a38265ce4d65aa35b54e01a2a906', 'request_id': 'req-c8834a7d-9478-401e-97ca-9541e7cbb723'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:04.115700 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-c8834a7d-9478-401e-97ca-9541e7cbb723 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33f9a38265ce4d65aa35b54e01a2a906', 'request_id': 'req-c8834a7d-9478-401e-97ca-9541e7cbb723'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:04.143811 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-c8834a7d-9478-401e-97ca-9541e7cbb723 admin admin] The token cache is being invalidate because project 33f9a38265ce4d65aa35b54e01a2a906 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:41:04.145578 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-c8834a7d-9478-401e-97ca-9541e7cbb723 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:04.148755 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-c8834a7d-9478-401e-97ca-9541e7cbb723 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:04.156692 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 884/1774] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:41:03 2022] DELETE /identity/v3/projects/33f9a38265ce4d65aa35b54e01a2a906 => generated 0 bytes in 750 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:41:04.161258 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55227e45-51f8-4011-a662-c38d0bf7e031 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:04.161843 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55227e45-51f8-4011-a662-c38d0bf7e031 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:04.162214 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55227e45-51f8-4011-a662-c38d0bf7e031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:04.238552 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-55227e45-51f8-4011-a662-c38d0bf7e031 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:41:04.245744 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-55227e45-51f8-4011-a662-c38d0bf7e031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:04.628535 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 891/1775] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:41:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 469 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:41:04.638124 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674cbd35-d782-468f-914b-d5a719a5de44 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:04.638630 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674cbd35-d782-468f-914b-d5a719a5de44 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:04.639880 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674cbd35-d782-468f-914b-d5a719a5de44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:04.643125 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:41:04.722736 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-674cbd35-d782-468f-914b-d5a719a5de44 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:41:04.739108 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-674cbd35-d782-468f-914b-d5a719a5de44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:04.785605 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 885/1776] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:41:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:41:04.788751 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55227e45-51f8-4011-a662-c38d0bf7e031 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:04.804732 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55227e45-51f8-4011-a662-c38d0bf7e031 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:04.838898 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d87f7f6-6d0a-4fc0-acf4-efec0e3d84de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:04.845583 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d87f7f6-6d0a-4fc0-acf4-efec0e3d84de admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:04.846645 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d87f7f6-6d0a-4fc0-acf4-efec0e3d84de admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:04.847432 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d87f7f6-6d0a-4fc0-acf4-efec0e3d84de admin admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:04.848509 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d87f7f6-6d0a-4fc0-acf4-efec0e3d84de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:41:04.849587 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d87f7f6-6d0a-4fc0-acf4-efec0e3d84de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:04.864315 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d87f7f6-6d0a-4fc0-acf4-efec0e3d84de admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:04.932090 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 892/1777] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 24 06:41:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:41:04.933785 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674cbd35-d782-468f-914b-d5a719a5de44 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:04.970620 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674cbd35-d782-468f-914b-d5a719a5de44 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:04.981836 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0ac5f2-68a5-4def-a3b1-9355653ae0e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:04.983099 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0ac5f2-68a5-4def-a3b1-9355653ae0e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:04.983546 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0ac5f2-68a5-4def-a3b1-9355653ae0e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:05.001112 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0ac5f2-68a5-4def-a3b1-9355653ae0e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:05.002485 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0ac5f2-68a5-4def-a3b1-9355653ae0e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:05.004850 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0ac5f2-68a5-4def-a3b1-9355653ae0e8 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:05.188521 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 886/1778] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:41:04 2022] POST /identity/v3/projects => generated 381 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:41:05.193148 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d87f7f6-6d0a-4fc0-acf4-efec0e3d84de admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:05.240359 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d87f7f6-6d0a-4fc0-acf4-efec0e3d84de admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:05.544257 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6268e768-7fbb-472b-8ae3-d0a3a8ec2857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:05.548904 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6268e768-7fbb-472b-8ae3-d0a3a8ec2857 admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:05.549807 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6268e768-7fbb-472b-8ae3-d0a3a8ec2857 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:05.550736 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6268e768-7fbb-472b-8ae3-d0a3a8ec2857 admin admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:05.557596 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6268e768-7fbb-472b-8ae3-d0a3a8ec2857 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:05.564469 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6268e768-7fbb-472b-8ae3-d0a3a8ec2857 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:05.581696 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0ac5f2-68a5-4def-a3b1-9355653ae0e8 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:05.583548 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-0c0ac5f2-68a5-4def-a3b1-9355653ae0e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:05.612485 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0ac5f2-68a5-4def-a3b1-9355653ae0e8 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:05.665109 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b5ec5f-f41d-47a9-8ed1-2ed67ed1c1db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:05.665984 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b5ec5f-f41d-47a9-8ed1-2ed67ed1c1db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:05.666154 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b5ec5f-f41d-47a9-8ed1-2ed67ed1c1db admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:05.666312 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b5ec5f-f41d-47a9-8ed1-2ed67ed1c1db admin admin] PATH_INFO: `/v3/projects/fa545d11e3f04f2488f53e6f7d5873b8` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:05.672216 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:41:05.677584 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b5ec5f-f41d-47a9-8ed1-2ed67ed1c1db admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa545d11e3f04f2488f53e6f7d5873b8)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:05.693163 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b5ec5f-f41d-47a9-8ed1-2ed67ed1c1db admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:05.767887 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 893/1779] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:41:05 2022] POST /identity/v3/users => generated 291 bytes in 575 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:41:05.775360 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6268e768-7fbb-472b-8ae3-d0a3a8ec2857 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:05.825670 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6268e768-7fbb-472b-8ae3-d0a3a8ec2857 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:05.847909 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ddc9a3-6640-42e8-9611-cff8830cbc0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:05.849777 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ddc9a3-6640-42e8-9611-cff8830cbc0a admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:05.850727 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ddc9a3-6640-42e8-9611-cff8830cbc0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:05.851491 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ddc9a3-6640-42e8-9611-cff8830cbc0a admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:05.865894 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ddc9a3-6640-42e8-9611-cff8830cbc0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:05.873947 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ddc9a3-6640-42e8-9611-cff8830cbc0a admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:05.961644 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 894/1780] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:41:05 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:41:05.966132 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ddc9a3-6640-42e8-9611-cff8830cbc0a admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:05.990616 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-83b5ec5f-f41d-47a9-8ed1-2ed67ed1c1db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa545d11e3f04f2488f53e6f7d5873b8', 'request_id': 'req-83b5ec5f-f41d-47a9-8ed1-2ed67ed1c1db'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:06.018819 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ddc9a3-6640-42e8-9611-cff8830cbc0a admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:06.022058 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-83b5ec5f-f41d-47a9-8ed1-2ed67ed1c1db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa545d11e3f04f2488f53e6f7d5873b8', 'request_id': 'req-83b5ec5f-f41d-47a9-8ed1-2ed67ed1c1db'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:06.252904 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-83b5ec5f-f41d-47a9-8ed1-2ed67ed1c1db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa545d11e3f04f2488f53e6f7d5873b8', 'request_id': 'req-83b5ec5f-f41d-47a9-8ed1-2ed67ed1c1db'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:06.279367 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3803800-2f9a-4afe-99c7-f2ac202451d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:06.280936 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3803800-2f9a-4afe-99c7-f2ac202451d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:06.281669 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3803800-2f9a-4afe-99c7-f2ac202451d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:06.281952 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3803800-2f9a-4afe-99c7-f2ac202451d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:06.291756 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3803800-2f9a-4afe-99c7-f2ac202451d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:06.300455 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3803800-2f9a-4afe-99c7-f2ac202451d7 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:06.306585 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-83b5ec5f-f41d-47a9-8ed1-2ed67ed1c1db admin admin] The token cache is being invalidate because project fa545d11e3f04f2488f53e6f7d5873b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:41:06.307081 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-83b5ec5f-f41d-47a9-8ed1-2ed67ed1c1db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:06.314067 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-83b5ec5f-f41d-47a9-8ed1-2ed67ed1c1db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:06.324881 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 887/1781] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:41:05 2022] DELETE /identity/v3/projects/fa545d11e3f04f2488f53e6f7d5873b8 => generated 0 bytes in 741 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:41:06.331050 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a65d6d2-5334-4667-8726-4e4c57faafc8 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:06.331551 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a65d6d2-5334-4667-8726-4e4c57faafc8 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:06.332067 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a65d6d2-5334-4667-8726-4e4c57faafc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:06.389852 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 895/1782] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:41:05 2022] GET /identity/v3/roles => generated 1420 bytes in 427 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:41:06.391837 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3803800-2f9a-4afe-99c7-f2ac202451d7 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:06.402248 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-f3803800-2f9a-4afe-99c7-f2ac202451d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:06.456924 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-0a65d6d2-5334-4667-8726-4e4c57faafc8 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:41:06.466767 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-0a65d6d2-5334-4667-8726-4e4c57faafc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:06.498128 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3803800-2f9a-4afe-99c7-f2ac202451d7 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:06.521566 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 888/1783] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:41:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:41:06.534579 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0429b71b-e1f6-4ed8-b47d-82db39b9d32f None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:06.534579 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0429b71b-e1f6-4ed8-b47d-82db39b9d32f None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:06.534579 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0429b71b-e1f6-4ed8-b47d-82db39b9d32f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:06.553368 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4006845e-20a4-49b9-85e3-8f340e7412f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:06.558347 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4006845e-20a4-49b9-85e3-8f340e7412f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:06.562213 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4006845e-20a4-49b9-85e3-8f340e7412f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:06.568357 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4006845e-20a4-49b9-85e3-8f340e7412f7 admin admin] PATH_INFO: `/v3/projects/63755f203bf04a66bc9b976116fd04ce/users/f190548aedf2489cb1a1b499756f6638/roles/e1f9b167c51c4a388cbc9594db8a465d` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:06.616212 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-0429b71b-e1f6-4ed8-b47d-82db39b9d32f None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:41:06.619389 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-0429b71b-e1f6-4ed8-b47d-82db39b9d32f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:06.658235 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4006845e-20a4-49b9-85e3-8f340e7412f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63755f203bf04a66bc9b976116fd04ce, user_id=f190548aedf2489cb1a1b499756f6638, role_id=e1f9b167c51c4a388cbc9594db8a465d)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:06.668083 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 889/1784] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:41:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:41:06.678312 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4006845e-20a4-49b9-85e3-8f340e7412f7 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:06.678900 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0429b71b-e1f6-4ed8-b47d-82db39b9d32f None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:06.682753 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0429b71b-e1f6-4ed8-b47d-82db39b9d32f None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:06.690434 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ef013d-41de-4f2b-9ca0-04ba9e4828f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:06.691077 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ef013d-41de-4f2b-9ca0-04ba9e4828f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:06.691374 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ef013d-41de-4f2b-9ca0-04ba9e4828f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:06.691631 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ef013d-41de-4f2b-9ca0-04ba9e4828f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:06.692395 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ef013d-41de-4f2b-9ca0-04ba9e4828f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:41:06.692708 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ef013d-41de-4f2b-9ca0-04ba9e4828f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:06.694833 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ef013d-41de-4f2b-9ca0-04ba9e4828f2 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:06.756570 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 890/1785] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 24 06:41:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:41:06.758131 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33312e44-65e7-4b94-a3c2-eb42bd6e0aca None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:06.758444 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33312e44-65e7-4b94-a3c2-eb42bd6e0aca None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:06.758714 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33312e44-65e7-4b94-a3c2-eb42bd6e0aca None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:06.766377 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 891/1786] 10.222.0.47 () {54 vars in 921 bytes} [Wed Aug 24 06:41:06 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:41:06.776788 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33312e44-65e7-4b94-a3c2-eb42bd6e0aca None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:06.821332 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33312e44-65e7-4b94-a3c2-eb42bd6e0aca None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:06.841622 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73636b21-0f6e-4012-84dd-9ccd8e77d2e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:06.842649 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73636b21-0f6e-4012-84dd-9ccd8e77d2e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:06.843016 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73636b21-0f6e-4012-84dd-9ccd8e77d2e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:06.843606 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73636b21-0f6e-4012-84dd-9ccd8e77d2e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:06.844337 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73636b21-0f6e-4012-84dd-9ccd8e77d2e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:06.849697 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73636b21-0f6e-4012-84dd-9ccd8e77d2e3 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:06.911783 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 896/1787] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:41:06 2022] PUT /identity/v3/projects/63755f203bf04a66bc9b976116fd04ce/users/f190548aedf2489cb1a1b499756f6638/roles/e1f9b167c51c4a388cbc9594db8a465d => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:41:06.911783 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ff76f2-2374-4050-84a6-e66604e8bba0 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:06.911783 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ff76f2-2374-4050-84a6-e66604e8bba0 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:06.911783 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ff76f2-2374-4050-84a6-e66604e8bba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:06.955032 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-72ff76f2-2374-4050-84a6-e66604e8bba0 None None] MFA Rules not processed for user `1f4f49c1b08c460f8ba6c451c9c92c2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:41:06.956596 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-72ff76f2-2374-4050-84a6-e66604e8bba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:07.105892 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 897/1788] 10.222.0.47 () {58 vars in 1018 bytes} [Wed Aug 24 06:41:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:41:07.107377 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ff76f2-2374-4050-84a6-e66604e8bba0 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:07.111463 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-72ff76f2-2374-4050-84a6-e66604e8bba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:07.243240 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ff76f2-2374-4050-84a6-e66604e8bba0 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:07.254890 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00fb83c-5262-4585-b21a-0c5590d78b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:07.256660 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00fb83c-5262-4585-b21a-0c5590d78b3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:07.257455 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00fb83c-5262-4585-b21a-0c5590d78b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:07.258124 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00fb83c-5262-4585-b21a-0c5590d78b3a admin admin] PATH_INFO: `/v3/users/c0dc621c3c6a41ae81ff4c43bb8f0307` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:07.261560 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00fb83c-5262-4585-b21a-0c5590d78b3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0dc621c3c6a41ae81ff4c43bb8f0307)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:07.266107 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00fb83c-5262-4585-b21a-0c5590d78b3a admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:07.306419 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 892/1789] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:41:06 2022] POST /identity/v3/projects => generated 407 bytes in 539 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:41:07.307558 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcae8bc-7dfe-48be-8d1a-a158edab0959 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:07.307849 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcae8bc-7dfe-48be-8d1a-a158edab0959 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:07.308900 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcae8bc-7dfe-48be-8d1a-a158edab0959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:07.327548 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:41:07.358816 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-5fcae8bc-7dfe-48be-8d1a-a158edab0959 None None] MFA Rules not processed for user `f190548aedf2489cb1a1b499756f6638`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:41:07.361146 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-5fcae8bc-7dfe-48be-8d1a-a158edab0959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:07.478770 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 893/1790] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:41:07 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:41:07.478770 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcae8bc-7dfe-48be-8d1a-a158edab0959 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:07.480276 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-5fcae8bc-7dfe-48be-8d1a-a158edab0959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:07.594783 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-f00fb83c-5262-4585-b21a-0c5590d78b3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0dc621c3c6a41ae81ff4c43bb8f0307'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:07.596420 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-f00fb83c-5262-4585-b21a-0c5590d78b3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0dc621c3c6a41ae81ff4c43bb8f0307'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:07.597433 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-f00fb83c-5262-4585-b21a-0c5590d78b3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0dc621c3c6a41ae81ff4c43bb8f0307'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:07.599045 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcae8bc-7dfe-48be-8d1a-a158edab0959 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:07.607358 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73752d8-c4b7-43c4-8cd4-f494e2e53542 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4f49c1b08c460f8ba6c451c9c92c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:07.608623 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73752d8-c4b7-43c4-8cd4-f494e2e53542 service nova] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:07.608623 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73752d8-c4b7-43c4-8cd4-f494e2e53542 service nova] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:07.610620 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73752d8-c4b7-43c4-8cd4-f494e2e53542 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:07.611654 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-a73752d8-c4b7-43c4-8cd4-f494e2e53542 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:07.628609 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73752d8-c4b7-43c4-8cd4-f494e2e53542 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:07.631480 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73752d8-c4b7-43c4-8cd4-f494e2e53542 service nova] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:07.645851 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 894/1791] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:41:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:41:07.647401 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73752d8-c4b7-43c4-8cd4-f494e2e53542 service nova] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:07.649793 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-a73752d8-c4b7-43c4-8cd4-f494e2e53542 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:07.681096 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-f00fb83c-5262-4585-b21a-0c5590d78b3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0dc621c3c6a41ae81ff4c43bb8f0307'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:07.707230 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-f00fb83c-5262-4585-b21a-0c5590d78b3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0dc621c3c6a41ae81ff4c43bb8f0307'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:07.738594 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 898/1792] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:41:07 2022] DELETE /identity/v3/users/c0dc621c3c6a41ae81ff4c43bb8f0307 => generated 0 bytes in 627 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:41:07.738594 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00fb83c-5262-4585-b21a-0c5590d78b3a admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:07.739665 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-f00fb83c-5262-4585-b21a-0c5590d78b3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:07.765291 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73752d8-c4b7-43c4-8cd4-f494e2e53542 service nova] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:07.866432 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cfc0b7-1785-4b69-8810-f9227d0f30cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:07.867462 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cfc0b7-1785-4b69-8810-f9227d0f30cb admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:07.867946 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cfc0b7-1785-4b69-8810-f9227d0f30cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:07.868392 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cfc0b7-1785-4b69-8810-f9227d0f30cb admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:07.869238 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cfc0b7-1785-4b69-8810-f9227d0f30cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:07.873735 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00fb83c-5262-4585-b21a-0c5590d78b3a admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:07.874038 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cfc0b7-1785-4b69-8810-f9227d0f30cb admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:07.880727 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af67fe57-99b9-423d-b0d4-84c47bd9a538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:07.881370 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af67fe57-99b9-423d-b0d4-84c47bd9a538 admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:07.881853 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af67fe57-99b9-423d-b0d4-84c47bd9a538 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:07.882513 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af67fe57-99b9-423d-b0d4-84c47bd9a538 admin admin] PATH_INFO: `/v3/projects` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:07.883669 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af67fe57-99b9-423d-b0d4-84c47bd9a538 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:07.887792 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af67fe57-99b9-423d-b0d4-84c47bd9a538 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:08.065723 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 895/1793] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:41:07 2022] POST /identity/v3/users => generated 304 bytes in 419 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:41:08.096853 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df270863-8046-4122-adf1-d8232729ab43 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:08.096853 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df270863-8046-4122-adf1-d8232729ab43 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:08.096853 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df270863-8046-4122-adf1-d8232729ab43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:08.096853 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-df270863-8046-4122-adf1-d8232729ab43 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:41:08.096853 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-df270863-8046-4122-adf1-d8232729ab43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:08.105364 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 896/1794] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:41:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:41:08.105898 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df270863-8046-4122-adf1-d8232729ab43 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:08.117503 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df270863-8046-4122-adf1-d8232729ab43 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:08.126693 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddb7501-5f74-4d2e-b55a-5d78fa8d59ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:08.127660 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddb7501-5f74-4d2e-b55a-5d78fa8d59ae admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:08.128292 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddb7501-5f74-4d2e-b55a-5d78fa8d59ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:08.128888 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddb7501-5f74-4d2e-b55a-5d78fa8d59ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:08.131202 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddb7501-5f74-4d2e-b55a-5d78fa8d59ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:08.133949 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddb7501-5f74-4d2e-b55a-5d78fa8d59ae admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:08.157023 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 897/1795] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:41:08 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:41:08.172437 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddb7501-5f74-4d2e-b55a-5d78fa8d59ae admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:08.172648 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 899/1796] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:41:07 2022] POST /identity/v3/projects => generated 379 bytes in 427 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:41:08.172802 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-cddb7501-5f74-4d2e-b55a-5d78fa8d59ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:08.174311 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af67fe57-99b9-423d-b0d4-84c47bd9a538 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:08.258828 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddb7501-5f74-4d2e-b55a-5d78fa8d59ae admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:08.294803 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af67fe57-99b9-423d-b0d4-84c47bd9a538 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:08.324582 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7affcb9-fe53-419c-bb95-52a1a80d4e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:08.326375 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7affcb9-fe53-419c-bb95-52a1a80d4e64 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:08.326850 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7affcb9-fe53-419c-bb95-52a1a80d4e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:08.327376 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7affcb9-fe53-419c-bb95-52a1a80d4e64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:08.329150 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7affcb9-fe53-419c-bb95-52a1a80d4e64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:08.333910 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7affcb9-fe53-419c-bb95-52a1a80d4e64 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:08.353412 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 900/1797] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:41:08 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:41:08.354421 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7affcb9-fe53-419c-bb95-52a1a80d4e64 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:08.378995 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7affcb9-fe53-419c-bb95-52a1a80d4e64 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:08.380122 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86c7990-b957-422c-88b2-362e1d7b32cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:08.381475 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86c7990-b957-422c-88b2-362e1d7b32cb service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:08.382109 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86c7990-b957-422c-88b2-362e1d7b32cb service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:08.382697 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86c7990-b957-422c-88b2-362e1d7b32cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:08.385403 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e15ea81-a982-4868-b1ff-101872fb4570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:08.386485 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e15ea81-a982-4868-b1ff-101872fb4570 admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:08.386927 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e15ea81-a982-4868-b1ff-101872fb4570 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:08.387216 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e15ea81-a982-4868-b1ff-101872fb4570 admin admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:08.390015 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e15ea81-a982-4868-b1ff-101872fb4570 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:08.393923 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e15ea81-a982-4868-b1ff-101872fb4570 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:08.400156 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86c7990-b957-422c-88b2-362e1d7b32cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:08.401770 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86c7990-b957-422c-88b2-362e1d7b32cb service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:08.434319 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 898/1798] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:41:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:41:08.434319 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86c7990-b957-422c-88b2-362e1d7b32cb service neutron] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:08.441433 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86c7990-b957-422c-88b2-362e1d7b32cb service neutron] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:08.460804 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fbc229-70f0-4f8f-a4d0-1a1685b33e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:08.464840 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fbc229-70f0-4f8f-a4d0-1a1685b33e90 admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:08.464840 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fbc229-70f0-4f8f-a4d0-1a1685b33e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:08.464840 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fbc229-70f0-4f8f-a4d0-1a1685b33e90 admin admin] PATH_INFO: `/v3/projects/952767ebbfa548649beff4719381b354/users/b5abf796f042439bb93751971206a963/roles/e1f9b167c51c4a388cbc9594db8a465d` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:08.527157 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fbc229-70f0-4f8f-a4d0-1a1685b33e90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=952767ebbfa548649beff4719381b354, user_id=b5abf796f042439bb93751971206a963, role_id=e1f9b167c51c4a388cbc9594db8a465d)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:08.533685 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fbc229-70f0-4f8f-a4d0-1a1685b33e90 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:08.570701 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 901/1799] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:41:08 2022] POST /identity/v3/users => generated 296 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:41:08.576376 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e15ea81-a982-4868-b1ff-101872fb4570 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:08.610323 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e15ea81-a982-4868-b1ff-101872fb4570 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:08.626697 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 899/1800] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:41:08 2022] PUT /identity/v3/projects/952767ebbfa548649beff4719381b354/users/b5abf796f042439bb93751971206a963/roles/e1f9b167c51c4a388cbc9594db8a465d => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:41:08.632252 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87735b52-e0eb-4e4d-a64e-a0ac7b336aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:08.633277 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87735b52-e0eb-4e4d-a64e-a0ac7b336aaa admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:08.633771 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87735b52-e0eb-4e4d-a64e-a0ac7b336aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:08.634262 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87735b52-e0eb-4e4d-a64e-a0ac7b336aaa admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:08.635113 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87735b52-e0eb-4e4d-a64e-a0ac7b336aaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:08.636939 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87735b52-e0eb-4e4d-a64e-a0ac7b336aaa admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:08.643029 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2d31c1-7a33-4480-98c8-f46c382d453b None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:08.643738 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2d31c1-7a33-4480-98c8-f46c382d453b None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:08.644213 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2d31c1-7a33-4480-98c8-f46c382d453b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:08.669369 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 902/1801] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:41:08 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:41:08.676008 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87735b52-e0eb-4e4d-a64e-a0ac7b336aaa admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:08.697781 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-fa2d31c1-7a33-4480-98c8-f46c382d453b None None] MFA Rules not processed for user `b5abf796f042439bb93751971206a963`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:41:08.706169 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-fa2d31c1-7a33-4480-98c8-f46c382d453b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:08.706430 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87735b52-e0eb-4e4d-a64e-a0ac7b336aaa admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:08.851732 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d00ba84-ea51-4dd3-8176-b102f52cf729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:08.853558 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d00ba84-ea51-4dd3-8176-b102f52cf729 admin admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:08.854144 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d00ba84-ea51-4dd3-8176-b102f52cf729 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:08.854692 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d00ba84-ea51-4dd3-8176-b102f52cf729 admin admin] PATH_INFO: `/v3/projects/a0f8dc5f327c41d9a230399368642d12/users/933190d09f0647d689d7f00742efbb59/roles/486144ad1bcd417e86acdbc1d14327bf` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:08.870061 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 900/1802] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:41:08 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Aug 24 06:41:08.877617 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2d31c1-7a33-4480-98c8-f46c382d453b None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:08.879867 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d00ba84-ea51-4dd3-8176-b102f52cf729 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0f8dc5f327c41d9a230399368642d12, user_id=933190d09f0647d689d7f00742efbb59, role_id=486144ad1bcd417e86acdbc1d14327bf)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:08.885680 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d00ba84-ea51-4dd3-8176-b102f52cf729 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:08.889763 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2d31c1-7a33-4480-98c8-f46c382d453b None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:08.895797 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a417595-f378-45e2-bb35-ed9b236af693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:08.896475 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a417595-f378-45e2-bb35-ed9b236af693 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:08.896747 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a417595-f378-45e2-bb35-ed9b236af693 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:08.897225 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a417595-f378-45e2-bb35-ed9b236af693 admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:08.898036 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a417595-f378-45e2-bb35-ed9b236af693 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:08.900159 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a417595-f378-45e2-bb35-ed9b236af693 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:08.909377 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:41:08.989769 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 903/1803] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:41:08 2022] PUT /identity/v3/projects/a0f8dc5f327c41d9a230399368642d12/users/933190d09f0647d689d7f00742efbb59/roles/486144ad1bcd417e86acdbc1d14327bf => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:41:08.993433 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d00ba84-ea51-4dd3-8176-b102f52cf729 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:09.079463 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 901/1804] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:41:08 2022] POST /identity/v3/projects => generated 409 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:41:09.086649 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a417595-f378-45e2-bb35-ed9b236af693 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:09.096366 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d00ba84-ea51-4dd3-8176-b102f52cf729 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:09.204813 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:41:09.205335 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a417595-f378-45e2-bb35-ed9b236af693 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:09.214962 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ea5676-68bc-48e0-9c10-072739dfcd1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:09.215529 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3ca897-7797-4b34-b394-bf8e0989d040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:09.216160 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ea5676-68bc-48e0-9c10-072739dfcd1c admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:09.216571 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3ca897-7797-4b34-b394-bf8e0989d040 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:09.216765 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ea5676-68bc-48e0-9c10-072739dfcd1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:09.217963 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ea5676-68bc-48e0-9c10-072739dfcd1c admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:09.218346 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3ca897-7797-4b34-b394-bf8e0989d040 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:09.219009 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3ca897-7797-4b34-b394-bf8e0989d040 admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:09.219467 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ea5676-68bc-48e0-9c10-072739dfcd1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:09.220115 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3ca897-7797-4b34-b394-bf8e0989d040 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:09.222886 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3ca897-7797-4b34-b394-bf8e0989d040 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:09.223017 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ea5676-68bc-48e0-9c10-072739dfcd1c admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:09.243085 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 904/1805] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:41:08 2022] GET /identity/v3/roles => generated 1420 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:41:09.247567 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3ca897-7797-4b34-b394-bf8e0989d040 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:09.262446 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3ca897-7797-4b34-b394-bf8e0989d040 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:09.267942 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a2d775-9dc6-4ebd-9f76-0e77088b1f72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:09.268756 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a2d775-9dc6-4ebd-9f76-0e77088b1f72 admin admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:09.269030 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a2d775-9dc6-4ebd-9f76-0e77088b1f72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:09.269346 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a2d775-9dc6-4ebd-9f76-0e77088b1f72 admin admin] PATH_INFO: `/v3/projects/a0f8dc5f327c41d9a230399368642d12/users/933190d09f0647d689d7f00742efbb59/roles/486144ad1bcd417e86acdbc1d14327bf` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:09.271359 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a2d775-9dc6-4ebd-9f76-0e77088b1f72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0f8dc5f327c41d9a230399368642d12, user_id=933190d09f0647d689d7f00742efbb59, role_id=486144ad1bcd417e86acdbc1d14327bf)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:09.274168 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a2d775-9dc6-4ebd-9f76-0e77088b1f72 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:09.285104 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 905/1806] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:41:09 2022] PUT /identity/v3/projects/a0f8dc5f327c41d9a230399368642d12/users/933190d09f0647d689d7f00742efbb59/roles/486144ad1bcd417e86acdbc1d14327bf => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:41:09.296032 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58765515-5590-4afa-ad61-3f921546d083 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:09.296328 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58765515-5590-4afa-ad61-3f921546d083 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:09.296609 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58765515-5590-4afa-ad61-3f921546d083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:09.319206 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 902/1807] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:41:09 2022] POST /identity/v3/users => generated 311 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:41:09.323384 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ea5676-68bc-48e0-9c10-072739dfcd1c admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:09.331964 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-58765515-5590-4afa-ad61-3f921546d083 None None] MFA Rules not processed for user `933190d09f0647d689d7f00742efbb59`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:41:09.333509 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-58765515-5590-4afa-ad61-3f921546d083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:09.337756 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ea5676-68bc-48e0-9c10-072739dfcd1c admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:09.431702 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0c2459-77f0-4507-8f70-8bc8cc050606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:09.432691 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0c2459-77f0-4507-8f70-8bc8cc050606 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:09.432691 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0c2459-77f0-4507-8f70-8bc8cc050606 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:09.433007 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0c2459-77f0-4507-8f70-8bc8cc050606 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:09.433517 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0c2459-77f0-4507-8f70-8bc8cc050606 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:09.435173 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0c2459-77f0-4507-8f70-8bc8cc050606 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:09.439174 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 906/1808] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:41:09 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:41:09.439632 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58765515-5590-4afa-ad61-3f921546d083 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:09.440988 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-58765515-5590-4afa-ad61-3f921546d083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:09.454732 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 903/1809] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:41:09 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:41:09.454732 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0c2459-77f0-4507-8f70-8bc8cc050606 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:09.454984 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-5c0c2459-77f0-4507-8f70-8bc8cc050606 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:09.466750 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58765515-5590-4afa-ad61-3f921546d083 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:09.473334 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a910c680-11c6-4d36-b300-0e41f8e93a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:09.474104 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a910c680-11c6-4d36-b300-0e41f8e93a7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:09.474104 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a910c680-11c6-4d36-b300-0e41f8e93a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:09.474272 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a910c680-11c6-4d36-b300-0e41f8e93a7a admin admin] PATH_INFO: `/v3/projects/28b24aaeafe646739194e971d85444ac` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:09.475598 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a910c680-11c6-4d36-b300-0e41f8e93a7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=28b24aaeafe646739194e971d85444ac)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:09.478219 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a910c680-11c6-4d36-b300-0e41f8e93a7a admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:09.526465 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0c2459-77f0-4507-8f70-8bc8cc050606 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:09.530075 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810d5559-4594-43f0-8377-855bee30cdcb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36639dce4f97412f89d71367f650b901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:09.542915 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810d5559-4594-43f0-8377-855bee30cdcb service cinder] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:09.542915 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810d5559-4594-43f0-8377-855bee30cdcb service cinder] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:09.542915 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810d5559-4594-43f0-8377-855bee30cdcb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:09.542915 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810d5559-4594-43f0-8377-855bee30cdcb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:09.545439 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810d5559-4594-43f0-8377-855bee30cdcb service cinder] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:09.558763 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 904/1810] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:41:09 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:41:09.559689 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810d5559-4594-43f0-8377-855bee30cdcb service cinder] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:09.574642 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810d5559-4594-43f0-8377-855bee30cdcb service cinder] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:09.582761 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc63eba4-bf5e-4aac-8d87-6de8ae24db95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:09.585604 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc63eba4-bf5e-4aac-8d87-6de8ae24db95 admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:09.585604 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc63eba4-bf5e-4aac-8d87-6de8ae24db95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:09.585604 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc63eba4-bf5e-4aac-8d87-6de8ae24db95 admin admin] PATH_INFO: `/v3/projects/8514dfa63c584b99892f05f2099572b9/users/ef0d8a77a8ac41f0ad15c235e84a6024/roles/486144ad1bcd417e86acdbc1d14327bf` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:09.606627 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc63eba4-bf5e-4aac-8d87-6de8ae24db95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8514dfa63c584b99892f05f2099572b9, user_id=ef0d8a77a8ac41f0ad15c235e84a6024, role_id=486144ad1bcd417e86acdbc1d14327bf)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:09.609996 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc63eba4-bf5e-4aac-8d87-6de8ae24db95 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:09.790237 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 905/1811] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:41:09 2022] PUT /identity/v3/projects/8514dfa63c584b99892f05f2099572b9/users/ef0d8a77a8ac41f0ad15c235e84a6024/roles/486144ad1bcd417e86acdbc1d14327bf => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:41:09.802996 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-a910c680-11c6-4d36-b300-0e41f8e93a7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28b24aaeafe646739194e971d85444ac', 'request_id': 'req-a910c680-11c6-4d36-b300-0e41f8e93a7a'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:09.818339 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc63eba4-bf5e-4aac-8d87-6de8ae24db95 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:09.829810 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:41:09.934843 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc63eba4-bf5e-4aac-8d87-6de8ae24db95 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:10.007585 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-a910c680-11c6-4d36-b300-0e41f8e93a7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28b24aaeafe646739194e971d85444ac', 'request_id': 'req-a910c680-11c6-4d36-b300-0e41f8e93a7a'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:10.027561 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-a910c680-11c6-4d36-b300-0e41f8e93a7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28b24aaeafe646739194e971d85444ac', 'request_id': 'req-a910c680-11c6-4d36-b300-0e41f8e93a7a'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:10.046291 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb81145-ac5d-4ca1-91a5-e07080f6c103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:10.047084 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb81145-ac5d-4ca1-91a5-e07080f6c103 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:10.047372 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb81145-ac5d-4ca1-91a5-e07080f6c103 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:10.047638 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb81145-ac5d-4ca1-91a5-e07080f6c103 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:10.048480 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb81145-ac5d-4ca1-91a5-e07080f6c103 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:10.050581 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb81145-ac5d-4ca1-91a5-e07080f6c103 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:10.073278 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 906/1812] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:41:09 2022] GET /identity/v3/roles => generated 1420 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:41:10.075786 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb81145-ac5d-4ca1-91a5-e07080f6c103 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:10.077323 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-a910c680-11c6-4d36-b300-0e41f8e93a7a admin admin] The token cache is being invalidate because project 28b24aaeafe646739194e971d85444ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81723) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:41:10.077642 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-a910c680-11c6-4d36-b300-0e41f8e93a7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:10.108089 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-a910c680-11c6-4d36-b300-0e41f8e93a7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:10.113377 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 907/1813] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:41:09 2022] DELETE /identity/v3/projects/28b24aaeafe646739194e971d85444ac => generated 0 bytes in 674 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:41:10.119713 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a910c680-11c6-4d36-b300-0e41f8e93a7a admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:10.125398 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-a910c680-11c6-4d36-b300-0e41f8e93a7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:10.146191 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb81145-ac5d-4ca1-91a5-e07080f6c103 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:10.169038 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9523150c-22a9-49de-924b-5b1f9879c122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:10.176549 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9523150c-22a9-49de-924b-5b1f9879c122 admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:10.176549 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9523150c-22a9-49de-924b-5b1f9879c122 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:10.176549 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9523150c-22a9-49de-924b-5b1f9879c122 admin admin] PATH_INFO: `/v3/projects/8514dfa63c584b99892f05f2099572b9/users/ef0d8a77a8ac41f0ad15c235e84a6024/roles/486144ad1bcd417e86acdbc1d14327bf` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:10.185562 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a910c680-11c6-4d36-b300-0e41f8e93a7a admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:10.190854 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9523150c-22a9-49de-924b-5b1f9879c122 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8514dfa63c584b99892f05f2099572b9, user_id=ef0d8a77a8ac41f0ad15c235e84a6024, role_id=486144ad1bcd417e86acdbc1d14327bf)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:10.194017 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9523150c-22a9-49de-924b-5b1f9879c122 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:10.206781 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61590fd7-1159-47e6-91d7-cf24a1b0530b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:10.214193 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61590fd7-1159-47e6-91d7-cf24a1b0530b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:10.214193 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61590fd7-1159-47e6-91d7-cf24a1b0530b admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:10.214193 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61590fd7-1159-47e6-91d7-cf24a1b0530b admin admin] PATH_INFO: `/v3/users/5919707e716e4ef28b5fe800c29167cc` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:10.215054 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61590fd7-1159-47e6-91d7-cf24a1b0530b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5919707e716e4ef28b5fe800c29167cc)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:10.228424 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61590fd7-1159-47e6-91d7-cf24a1b0530b admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:10.258220 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 907/1814] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:41:10 2022] PUT /identity/v3/projects/8514dfa63c584b99892f05f2099572b9/users/ef0d8a77a8ac41f0ad15c235e84a6024/roles/486144ad1bcd417e86acdbc1d14327bf => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:41:10.270573 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e251d9-1e92-4d98-8637-0e6c57a6b3fc None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:10.270941 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e251d9-1e92-4d98-8637-0e6c57a6b3fc None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:10.271420 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e251d9-1e92-4d98-8637-0e6c57a6b3fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:10.419161 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-37e251d9-1e92-4d98-8637-0e6c57a6b3fc None None] MFA Rules not processed for user `ef0d8a77a8ac41f0ad15c235e84a6024`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:41:10.423180 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-37e251d9-1e92-4d98-8637-0e6c57a6b3fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:10.591309 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 908/1815] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:41:10 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:41:10.621301 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e251d9-1e92-4d98-8637-0e6c57a6b3fc None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:10.622466 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-37e251d9-1e92-4d98-8637-0e6c57a6b3fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:10.716581 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e251d9-1e92-4d98-8637-0e6c57a6b3fc None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:10.720601 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-61590fd7-1159-47e6-91d7-cf24a1b0530b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5919707e716e4ef28b5fe800c29167cc'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:10.722076 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-61590fd7-1159-47e6-91d7-cf24a1b0530b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5919707e716e4ef28b5fe800c29167cc'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:10.722774 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-61590fd7-1159-47e6-91d7-cf24a1b0530b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5919707e716e4ef28b5fe800c29167cc'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:10.725802 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a075d32f-731f-4189-9c18-688e35cfb45c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36639dce4f97412f89d71367f650b901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:10.727058 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a075d32f-731f-4189-9c18-688e35cfb45c service cinder] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:10.727709 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a075d32f-731f-4189-9c18-688e35cfb45c service cinder] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:10.728200 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a075d32f-731f-4189-9c18-688e35cfb45c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:10.731197 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-a075d32f-731f-4189-9c18-688e35cfb45c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:10.753241 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a075d32f-731f-4189-9c18-688e35cfb45c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:10.755352 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a075d32f-731f-4189-9c18-688e35cfb45c service cinder] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:10.770249 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 909/1816] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:41:10 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:41:10.829614 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-61590fd7-1159-47e6-91d7-cf24a1b0530b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5919707e716e4ef28b5fe800c29167cc'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:10.846637 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-61590fd7-1159-47e6-91d7-cf24a1b0530b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5919707e716e4ef28b5fe800c29167cc'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:10.867115 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 908/1817] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:41:10 2022] DELETE /identity/v3/users/5919707e716e4ef28b5fe800c29167cc => generated 0 bytes in 745 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:41:10.898292 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a075d32f-731f-4189-9c18-688e35cfb45c service cinder] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:10.900902 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-a075d32f-731f-4189-9c18-688e35cfb45c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:11.044637 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a075d32f-731f-4189-9c18-688e35cfb45c service cinder] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:11.151186 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434d7ea5-076e-400c-9884-eda467a7bc13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:11.157680 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d7ea5-076e-400c-9884-eda467a7bc13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:11.157680 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d7ea5-076e-400c-9884-eda467a7bc13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:11.157680 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434d7ea5-076e-400c-9884-eda467a7bc13 admin admin] PATH_INFO: `/v3/users/f190548aedf2489cb1a1b499756f6638` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:11.157680 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434d7ea5-076e-400c-9884-eda467a7bc13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f190548aedf2489cb1a1b499756f6638)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:11.161557 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434d7ea5-076e-400c-9884-eda467a7bc13 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:11.418721 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61590fd7-1159-47e6-91d7-cf24a1b0530b admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:11.418721 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-61590fd7-1159-47e6-91d7-cf24a1b0530b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:11.514212 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-434d7ea5-076e-400c-9884-eda467a7bc13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f190548aedf2489cb1a1b499756f6638'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:11.515186 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-434d7ea5-076e-400c-9884-eda467a7bc13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f190548aedf2489cb1a1b499756f6638'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:11.515989 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-434d7ea5-076e-400c-9884-eda467a7bc13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f190548aedf2489cb1a1b499756f6638'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:11.521461 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61590fd7-1159-47e6-91d7-cf24a1b0530b admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:11.527028 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-434d7ea5-076e-400c-9884-eda467a7bc13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f190548aedf2489cb1a1b499756f6638'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:11.527818 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e49b52-b25c-4a59-bbc6-685645e63808 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36639dce4f97412f89d71367f650b901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:11.528570 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e49b52-b25c-4a59-bbc6-685645e63808 service cinder] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:11.528950 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e49b52-b25c-4a59-bbc6-685645e63808 service cinder] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:11.529444 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e49b52-b25c-4a59-bbc6-685645e63808 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:11.532504 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-47e49b52-b25c-4a59-bbc6-685645e63808 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:11.585615 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:41:11.636051 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e49b52-b25c-4a59-bbc6-685645e63808 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:11.642121 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e49b52-b25c-4a59-bbc6-685645e63808 service cinder] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:11.659465 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 909/1818] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:41:11 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:41:11.660435 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e49b52-b25c-4a59-bbc6-685645e63808 service cinder] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:11.663661 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-47e49b52-b25c-4a59-bbc6-685645e63808 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:11.675281 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-434d7ea5-076e-400c-9884-eda467a7bc13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f190548aedf2489cb1a1b499756f6638'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:11.687754 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e49b52-b25c-4a59-bbc6-685645e63808 service cinder] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:11.694393 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f1eb92-2d2e-4b29-bb7b-4f34d9763883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:11.703862 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f1eb92-2d2e-4b29-bb7b-4f34d9763883 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:11.703862 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f1eb92-2d2e-4b29-bb7b-4f34d9763883 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:11.703862 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f1eb92-2d2e-4b29-bb7b-4f34d9763883 admin admin] PATH_INFO: `/v3/projects/fa25011931e84a71bd1b20a9f3766f54` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:11.703862 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f1eb92-2d2e-4b29-bb7b-4f34d9763883 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa25011931e84a71bd1b20a9f3766f54)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:11.703862 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f1eb92-2d2e-4b29-bb7b-4f34d9763883 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:11.719815 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 910/1819] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:41:10 2022] DELETE /identity/v3/users/f190548aedf2489cb1a1b499756f6638 => generated 0 bytes in 823 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:41:11.726228 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ec42d5-c48a-4b9e-9e06-3b9527dd2024 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:11.726934 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ec42d5-c48a-4b9e-9e06-3b9527dd2024 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:11.727470 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ec42d5-c48a-4b9e-9e06-3b9527dd2024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:11.753049 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-b1ec42d5-c48a-4b9e-9e06-3b9527dd2024 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:41:11.754677 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-b1ec42d5-c48a-4b9e-9e06-3b9527dd2024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:11.848606 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 911/1820] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:41:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:41:11.860611 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ec42d5-c48a-4b9e-9e06-3b9527dd2024 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:11.862283 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-b1ec42d5-c48a-4b9e-9e06-3b9527dd2024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:11.895608 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-01f1eb92-2d2e-4b29-bb7b-4f34d9763883 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa25011931e84a71bd1b20a9f3766f54', 'request_id': 'req-01f1eb92-2d2e-4b29-bb7b-4f34d9763883'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:11.939042 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ec42d5-c48a-4b9e-9e06-3b9527dd2024 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:12.002854 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7f0a6e-355d-44ab-b137-19c3b888fe94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:12.004015 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7f0a6e-355d-44ab-b137-19c3b888fe94 service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:12.004774 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7f0a6e-355d-44ab-b137-19c3b888fe94 service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:12.005317 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7f0a6e-355d-44ab-b137-19c3b888fe94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:12.063549 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-01f1eb92-2d2e-4b29-bb7b-4f34d9763883 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa25011931e84a71bd1b20a9f3766f54', 'request_id': 'req-01f1eb92-2d2e-4b29-bb7b-4f34d9763883'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:12.076480 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-01f1eb92-2d2e-4b29-bb7b-4f34d9763883 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa25011931e84a71bd1b20a9f3766f54', 'request_id': 'req-01f1eb92-2d2e-4b29-bb7b-4f34d9763883'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:12.092495 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-01f1eb92-2d2e-4b29-bb7b-4f34d9763883 admin admin] The token cache is being invalidate because project fa25011931e84a71bd1b20a9f3766f54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81723) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:41:12.093623 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-01f1eb92-2d2e-4b29-bb7b-4f34d9763883 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:12.094613 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-01f1eb92-2d2e-4b29-bb7b-4f34d9763883 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:12.096403 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 910/1821] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:41:11 2022] DELETE /identity/v3/projects/fa25011931e84a71bd1b20a9f3766f54 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:41:12.097954 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7f0a6e-355d-44ab-b137-19c3b888fe94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:12.099472 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7f0a6e-355d-44ab-b137-19c3b888fe94 service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:12.102842 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00c469f-b6fb-49b2-b404-76ad7f799dd0 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:12.103119 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00c469f-b6fb-49b2-b404-76ad7f799dd0 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:12.103401 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00c469f-b6fb-49b2-b404-76ad7f799dd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:12.113687 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 912/1822] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:41:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:41:12.128893 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-d00c469f-b6fb-49b2-b404-76ad7f799dd0 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:41:12.130200 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-d00c469f-b6fb-49b2-b404-76ad7f799dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:12.138584 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 911/1823] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:41:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:41:12.154161 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4703b896-0b6e-41f0-a8d4-f487ef5d73b0 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:12.158180 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4703b896-0b6e-41f0-a8d4-f487ef5d73b0 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:12.158180 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4703b896-0b6e-41f0-a8d4-f487ef5d73b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:12.183224 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:41:12.193811 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-4703b896-0b6e-41f0-a8d4-f487ef5d73b0 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:41:12.195226 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-4703b896-0b6e-41f0-a8d4-f487ef5d73b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:12.210365 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 913/1824] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:41:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:41:12.214212 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00c469f-b6fb-49b2-b404-76ad7f799dd0 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:12.227452 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00c469f-b6fb-49b2-b404-76ad7f799dd0 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:12.241733 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b3058a-f894-49e0-9607-7b0289ee20e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:12.242374 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b3058a-f894-49e0-9607-7b0289ee20e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:12.242656 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b3058a-f894-49e0-9607-7b0289ee20e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:12.242931 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b3058a-f894-49e0-9607-7b0289ee20e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:12.243609 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b3058a-f894-49e0-9607-7b0289ee20e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:41:12.243925 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b3058a-f894-49e0-9607-7b0289ee20e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:12.246014 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b3058a-f894-49e0-9607-7b0289ee20e5 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:12.267206 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 912/1825] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 24 06:41:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:41:12.268714 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4703b896-0b6e-41f0-a8d4-f487ef5d73b0 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:12.283061 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4703b896-0b6e-41f0-a8d4-f487ef5d73b0 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:12.291186 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a911c18-d6e6-4aed-900e-1a823e8abfe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:12.292075 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a911c18-d6e6-4aed-900e-1a823e8abfe8 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:12.292550 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a911c18-d6e6-4aed-900e-1a823e8abfe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:12.292942 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a911c18-d6e6-4aed-900e-1a823e8abfe8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:12.293716 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a911c18-d6e6-4aed-900e-1a823e8abfe8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:12.296914 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a911c18-d6e6-4aed-900e-1a823e8abfe8 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:12.511905 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 914/1826] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:41:12 2022] POST /identity/v3/projects => generated 367 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:41:12.518120 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b3058a-f894-49e0-9607-7b0289ee20e5 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:12.543782 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b3058a-f894-49e0-9607-7b0289ee20e5 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:12.755042 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bd94db-aef1-4132-9124-b69002995763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:12.755780 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bd94db-aef1-4132-9124-b69002995763 admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:12.758338 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bd94db-aef1-4132-9124-b69002995763 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:12.758338 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bd94db-aef1-4132-9124-b69002995763 admin admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:12.758338 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bd94db-aef1-4132-9124-b69002995763 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:12.762106 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bd94db-aef1-4132-9124-b69002995763 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:13.102355 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a911c18-d6e6-4aed-900e-1a823e8abfe8 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:13.105330 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-0a911c18-d6e6-4aed-900e-1a823e8abfe8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:13.129241 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a911c18-d6e6-4aed-900e-1a823e8abfe8 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:13.134347 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 913/1827] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:41:12 2022] POST /identity/v3/users => generated 284 bytes in 617 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:41:13.136153 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bd94db-aef1-4132-9124-b69002995763 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:13.163615 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bd94db-aef1-4132-9124-b69002995763 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:13.164113 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839fed34-183f-4630-ba62-4c17d301cb31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:13.165590 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839fed34-183f-4630-ba62-4c17d301cb31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:13.166678 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839fed34-183f-4630-ba62-4c17d301cb31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:13.167089 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839fed34-183f-4630-ba62-4c17d301cb31 admin admin] PATH_INFO: `/v3/projects/63755f203bf04a66bc9b976116fd04ce` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:13.170768 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839fed34-183f-4630-ba62-4c17d301cb31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63755f203bf04a66bc9b976116fd04ce)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:13.177514 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64775052-a325-4080-9189-7b48ac5c8a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:13.178657 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64775052-a325-4080-9189-7b48ac5c8a29 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:13.178940 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64775052-a325-4080-9189-7b48ac5c8a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:13.179495 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64775052-a325-4080-9189-7b48ac5c8a29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:13.180843 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64775052-a325-4080-9189-7b48ac5c8a29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:13.183128 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839fed34-183f-4630-ba62-4c17d301cb31 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:13.183916 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64775052-a325-4080-9189-7b48ac5c8a29 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:13.205273 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 914/1828] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:41:13 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:41:13.351733 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64775052-a325-4080-9189-7b48ac5c8a29 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:13.351733 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64775052-a325-4080-9189-7b48ac5c8a29 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:13.351733 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a81d53-4f3f-485d-8e4e-15dd616bf7a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:13.351733 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a81d53-4f3f-485d-8e4e-15dd616bf7a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:13.352888 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a81d53-4f3f-485d-8e4e-15dd616bf7a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:13.352888 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a81d53-4f3f-485d-8e4e-15dd616bf7a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:13.352888 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a81d53-4f3f-485d-8e4e-15dd616bf7a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:13.352888 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a81d53-4f3f-485d-8e4e-15dd616bf7a2 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:13.352888 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 915/1829] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:41:13 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:41:13.356213 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a81d53-4f3f-485d-8e4e-15dd616bf7a2 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:13.367346 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a81d53-4f3f-485d-8e4e-15dd616bf7a2 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:13.372231 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd250f5-9a58-4886-b4b7-8e367a2a8321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:13.372964 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd250f5-9a58-4886-b4b7-8e367a2a8321 admin admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:13.373424 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd250f5-9a58-4886-b4b7-8e367a2a8321 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:13.373990 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd250f5-9a58-4886-b4b7-8e367a2a8321 admin admin] PATH_INFO: `/v3/projects/dbdecbd4f06542bea6308f575a89f0f9/users/6027122b2a8040d58e5bf9fc8a46400d/roles/e1f9b167c51c4a388cbc9594db8a465d` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:13.397355 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd250f5-9a58-4886-b4b7-8e367a2a8321 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbdecbd4f06542bea6308f575a89f0f9, user_id=6027122b2a8040d58e5bf9fc8a46400d, role_id=e1f9b167c51c4a388cbc9594db8a465d)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:13.400163 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd250f5-9a58-4886-b4b7-8e367a2a8321 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:13.963317 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 916/1830] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:41:13 2022] PUT /identity/v3/projects/dbdecbd4f06542bea6308f575a89f0f9/users/6027122b2a8040d58e5bf9fc8a46400d/roles/e1f9b167c51c4a388cbc9594db8a465d => generated 0 bytes in 608 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:41:13.989122 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e0c13-d884-434b-afe2-db0cbe21385b None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:13.989653 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e0c13-d884-434b-afe2-db0cbe21385b None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:13.990393 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e0c13-d884-434b-afe2-db0cbe21385b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:14.009560 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-839fed34-183f-4630-ba62-4c17d301cb31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63755f203bf04a66bc9b976116fd04ce', 'request_id': 'req-839fed34-183f-4630-ba62-4c17d301cb31'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:14.026996 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-839fed34-183f-4630-ba62-4c17d301cb31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63755f203bf04a66bc9b976116fd04ce', 'request_id': 'req-839fed34-183f-4630-ba62-4c17d301cb31'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:14.066333 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-532e0c13-d884-434b-afe2-db0cbe21385b None None] MFA Rules not processed for user `6027122b2a8040d58e5bf9fc8a46400d`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:41:14.068873 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-532e0c13-d884-434b-afe2-db0cbe21385b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:14.220903 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 917/1831] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:41:13 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:41:14.226142 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532e0c13-d884-434b-afe2-db0cbe21385b None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:14.316071 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532e0c13-d884-434b-afe2-db0cbe21385b None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:14.337480 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25cf1bd-6e27-44d8-914c-c36d857d6f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:14.338489 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25cf1bd-6e27-44d8-914c-c36d857d6f19 admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:14.338831 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25cf1bd-6e27-44d8-914c-c36d857d6f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:14.339015 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25cf1bd-6e27-44d8-914c-c36d857d6f19 admin admin] PATH_INFO: `/v3/projects` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:14.339728 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25cf1bd-6e27-44d8-914c-c36d857d6f19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:14.342401 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25cf1bd-6e27-44d8-914c-c36d857d6f19 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:14.451863 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-839fed34-183f-4630-ba62-4c17d301cb31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63755f203bf04a66bc9b976116fd04ce', 'request_id': 'req-839fed34-183f-4630-ba62-4c17d301cb31'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:14.451863 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-839fed34-183f-4630-ba62-4c17d301cb31 admin admin] The token cache is being invalidate because project 63755f203bf04a66bc9b976116fd04ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:41:14.451863 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-839fed34-183f-4630-ba62-4c17d301cb31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:14.451863 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-839fed34-183f-4630-ba62-4c17d301cb31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:14.451863 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 915/1832] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:41:13 2022] DELETE /identity/v3/projects/63755f203bf04a66bc9b976116fd04ce => generated 0 bytes in 1327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:41:14.453451 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839fed34-183f-4630-ba62-4c17d301cb31 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:14.453451 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-839fed34-183f-4630-ba62-4c17d301cb31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:14.460915 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839fed34-183f-4630-ba62-4c17d301cb31 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:14.467577 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c099293b-c04a-4a10-b00d-77b63f684cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:14.468620 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c099293b-c04a-4a10-b00d-77b63f684cb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:14.469040 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c099293b-c04a-4a10-b00d-77b63f684cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:14.469708 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c099293b-c04a-4a10-b00d-77b63f684cb2 admin admin] PATH_INFO: `/v3/users/933190d09f0647d689d7f00742efbb59` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:14.471697 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c099293b-c04a-4a10-b00d-77b63f684cb2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=933190d09f0647d689d7f00742efbb59)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:14.474029 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c099293b-c04a-4a10-b00d-77b63f684cb2 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:14.824312 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 918/1833] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:41:14 2022] POST /identity/v3/projects => generated 367 bytes in 598 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:41:14.829543 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25cf1bd-6e27-44d8-914c-c36d857d6f19 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:14.833031 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-d25cf1bd-6e27-44d8-914c-c36d857d6f19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:15.007915 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25cf1bd-6e27-44d8-914c-c36d857d6f19 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:15.013297 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942aa286-b588-4ad0-a54a-31e0fd4e6759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:15.015625 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942aa286-b588-4ad0-a54a-31e0fd4e6759 admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:15.016372 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942aa286-b588-4ad0-a54a-31e0fd4e6759 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:15.016977 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942aa286-b588-4ad0-a54a-31e0fd4e6759 admin admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:15.017849 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942aa286-b588-4ad0-a54a-31e0fd4e6759 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:15.021827 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942aa286-b588-4ad0-a54a-31e0fd4e6759 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:15.169336 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-c099293b-c04a-4a10-b00d-77b63f684cb2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '933190d09f0647d689d7f00742efbb59'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:15.169336 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-c099293b-c04a-4a10-b00d-77b63f684cb2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '933190d09f0647d689d7f00742efbb59'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:15.169336 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-c099293b-c04a-4a10-b00d-77b63f684cb2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '933190d09f0647d689d7f00742efbb59'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:15.169336 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-c099293b-c04a-4a10-b00d-77b63f684cb2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '933190d09f0647d689d7f00742efbb59'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:15.805423 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 919/1834] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:41:14 2022] POST /identity/v3/users => generated 290 bytes in 977 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:41:15.810230 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942aa286-b588-4ad0-a54a-31e0fd4e6759 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:15.813613 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-942aa286-b588-4ad0-a54a-31e0fd4e6759 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:15.825674 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-c099293b-c04a-4a10-b00d-77b63f684cb2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '933190d09f0647d689d7f00742efbb59'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:15.870844 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942aa286-b588-4ad0-a54a-31e0fd4e6759 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:15.877819 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4351887a-ddd5-4dc7-b172-abc9e094ad97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:15.878538 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4351887a-ddd5-4dc7-b172-abc9e094ad97 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:15.878972 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4351887a-ddd5-4dc7-b172-abc9e094ad97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:15.879345 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4351887a-ddd5-4dc7-b172-abc9e094ad97 admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:15.880112 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4351887a-ddd5-4dc7-b172-abc9e094ad97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:15.881964 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4351887a-ddd5-4dc7-b172-abc9e094ad97 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:15.894988 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 916/1835] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:41:14 2022] DELETE /identity/v3/users/933190d09f0647d689d7f00742efbb59 => generated 0 bytes in 1463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:41:15.898535 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 920/1836] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:41:15 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:41:15.907715 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c099293b-c04a-4a10-b00d-77b63f684cb2 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:16.027456 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c099293b-c04a-4a10-b00d-77b63f684cb2 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:16.064090 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2bd42d-8966-411a-a9d6-febe30f22aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:16.065881 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2bd42d-8966-411a-a9d6-febe30f22aac admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:16.066663 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2bd42d-8966-411a-a9d6-febe30f22aac admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:16.067435 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2bd42d-8966-411a-a9d6-febe30f22aac admin admin] PATH_INFO: `/v3/projects/18e22cf7cd3b468eab2928fc244490af/users/7a034871ca5749769822e57384b2429d/roles/486144ad1bcd417e86acdbc1d14327bf` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:16.104345 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2bd42d-8966-411a-a9d6-febe30f22aac admin admin] RBAC: Authorizing `identity:create_grant(project_id=18e22cf7cd3b468eab2928fc244490af, user_id=7a034871ca5749769822e57384b2429d, role_id=486144ad1bcd417e86acdbc1d14327bf)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:16.120179 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2bd42d-8966-411a-a9d6-febe30f22aac admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:16.405979 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 917/1837] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:41:15 2022] PUT /identity/v3/projects/18e22cf7cd3b468eab2928fc244490af/users/7a034871ca5749769822e57384b2429d/roles/486144ad1bcd417e86acdbc1d14327bf => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:41:16.407318 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4351887a-ddd5-4dc7-b172-abc9e094ad97 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:16.423005 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4351887a-ddd5-4dc7-b172-abc9e094ad97 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:16.536388 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16831287-3519-49f9-92db-47df0973e697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:16.537152 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16831287-3519-49f9-92db-47df0973e697 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:16.537538 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16831287-3519-49f9-92db-47df0973e697 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:16.537820 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16831287-3519-49f9-92db-47df0973e697 admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:16.539351 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16831287-3519-49f9-92db-47df0973e697 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:16.554154 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16831287-3519-49f9-92db-47df0973e697 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:16.591631 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 921/1838] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:41:16 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:41:16.602953 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2bd42d-8966-411a-a9d6-febe30f22aac admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:16.636731 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2bd42d-8966-411a-a9d6-febe30f22aac admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:16.652940 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0709c335-798a-45d7-a5d4-843b86358a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:16.653677 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0709c335-798a-45d7-a5d4-843b86358a46 admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:16.653999 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0709c335-798a-45d7-a5d4-843b86358a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:16.654291 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0709c335-798a-45d7-a5d4-843b86358a46 admin admin] PATH_INFO: `/v3/projects/18e22cf7cd3b468eab2928fc244490af/users/7a034871ca5749769822e57384b2429d/roles/486144ad1bcd417e86acdbc1d14327bf` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:16.657142 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0709c335-798a-45d7-a5d4-843b86358a46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18e22cf7cd3b468eab2928fc244490af, user_id=7a034871ca5749769822e57384b2429d, role_id=486144ad1bcd417e86acdbc1d14327bf)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:16.660944 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0709c335-798a-45d7-a5d4-843b86358a46 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:16.676486 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 918/1839] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:41:16 2022] PUT /identity/v3/projects/18e22cf7cd3b468eab2928fc244490af/users/7a034871ca5749769822e57384b2429d/roles/486144ad1bcd417e86acdbc1d14327bf => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:41:16.696361 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b746bc23-fd44-42f6-97de-afe7653405be None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:16.696871 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b746bc23-fd44-42f6-97de-afe7653405be None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:16.697235 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b746bc23-fd44-42f6-97de-afe7653405be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:16.769543 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-b746bc23-fd44-42f6-97de-afe7653405be None None] MFA Rules not processed for user `7a034871ca5749769822e57384b2429d`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:41:16.771946 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-b746bc23-fd44-42f6-97de-afe7653405be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:16.974440 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 922/1840] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:41:16 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:41:16.975396 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0709c335-798a-45d7-a5d4-843b86358a46 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:16.985737 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-0709c335-798a-45d7-a5d4-843b86358a46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:17.010100 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b746bc23-fd44-42f6-97de-afe7653405be None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:17.016209 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-b746bc23-fd44-42f6-97de-afe7653405be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:17.234428 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0709c335-798a-45d7-a5d4-843b86358a46 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:17.236385 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b746bc23-fd44-42f6-97de-afe7653405be None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:17.250264 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130c5530-a979-4dab-8fbd-d5cf125c41d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36639dce4f97412f89d71367f650b901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:17.252113 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130c5530-a979-4dab-8fbd-d5cf125c41d4 service cinder] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:17.252113 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130c5530-a979-4dab-8fbd-d5cf125c41d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:17.252462 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130c5530-a979-4dab-8fbd-d5cf125c41d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:17.253384 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287dd390-6856-44ec-8494-46e9c7b73d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:17.256869 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287dd390-6856-44ec-8494-46e9c7b73d03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:17.259053 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287dd390-6856-44ec-8494-46e9c7b73d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:17.259733 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287dd390-6856-44ec-8494-46e9c7b73d03 admin admin] PATH_INFO: `/v3/projects/a0f8dc5f327c41d9a230399368642d12` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:17.264957 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287dd390-6856-44ec-8494-46e9c7b73d03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0f8dc5f327c41d9a230399368642d12)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:17.283364 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287dd390-6856-44ec-8494-46e9c7b73d03 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:17.345067 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130c5530-a979-4dab-8fbd-d5cf125c41d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:17.348406 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130c5530-a979-4dab-8fbd-d5cf125c41d4 service cinder] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:17.393249 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 923/1841] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:41:17 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:41:17.602645 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-287dd390-6856-44ec-8494-46e9c7b73d03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0f8dc5f327c41d9a230399368642d12', 'request_id': 'req-287dd390-6856-44ec-8494-46e9c7b73d03'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:17.628157 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-287dd390-6856-44ec-8494-46e9c7b73d03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0f8dc5f327c41d9a230399368642d12', 'request_id': 'req-287dd390-6856-44ec-8494-46e9c7b73d03'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:17.949506 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-287dd390-6856-44ec-8494-46e9c7b73d03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0f8dc5f327c41d9a230399368642d12', 'request_id': 'req-287dd390-6856-44ec-8494-46e9c7b73d03'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:17.976118 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-287dd390-6856-44ec-8494-46e9c7b73d03 admin admin] The token cache is being invalidate because project a0f8dc5f327c41d9a230399368642d12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:41:17.976118 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-287dd390-6856-44ec-8494-46e9c7b73d03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:17.976118 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-287dd390-6856-44ec-8494-46e9c7b73d03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:17.976118 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 919/1842] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:41:16 2022] DELETE /identity/v3/projects/a0f8dc5f327c41d9a230399368642d12 => generated 0 bytes in 1002 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:41:17.983903 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d590d2-b23d-4627-9b1b-b578a43f556c None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:17.984499 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d590d2-b23d-4627-9b1b-b578a43f556c None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:17.984735 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d590d2-b23d-4627-9b1b-b578a43f556c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:18.030403 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:41:18.068175 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-f1d590d2-b23d-4627-9b1b-b578a43f556c None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:41:18.069471 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-f1d590d2-b23d-4627-9b1b-b578a43f556c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:18.251586 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 924/1843] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:41:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:41:18.260221 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8367264-68de-4acf-a22e-600e52ca0e00 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:18.261531 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8367264-68de-4acf-a22e-600e52ca0e00 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:18.265881 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8367264-68de-4acf-a22e-600e52ca0e00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:18.317716 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-d8367264-68de-4acf-a22e-600e52ca0e00 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:41:18.319015 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-d8367264-68de-4acf-a22e-600e52ca0e00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:18.357002 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 920/1844] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:41:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:41:18.362545 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d590d2-b23d-4627-9b1b-b578a43f556c None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:18.378432 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d590d2-b23d-4627-9b1b-b578a43f556c None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:18.389148 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bb26d3-8e6e-4964-95a8-e39a05a430b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:18.389148 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb26d3-8e6e-4964-95a8-e39a05a430b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:18.389148 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb26d3-8e6e-4964-95a8-e39a05a430b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:18.391699 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb26d3-8e6e-4964-95a8-e39a05a430b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:18.392554 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bb26d3-8e6e-4964-95a8-e39a05a430b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:41:18.393094 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bb26d3-8e6e-4964-95a8-e39a05a430b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:18.396055 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bb26d3-8e6e-4964-95a8-e39a05a430b1 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:18.419352 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 925/1845] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 24 06:41:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:41:18.428027 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8367264-68de-4acf-a22e-600e52ca0e00 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:18.439147 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8367264-68de-4acf-a22e-600e52ca0e00 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:18.450323 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37624fa5-bdaf-4f8e-9d2c-441529572b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:18.451818 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37624fa5-bdaf-4f8e-9d2c-441529572b4c admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:18.452215 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37624fa5-bdaf-4f8e-9d2c-441529572b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:18.452659 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37624fa5-bdaf-4f8e-9d2c-441529572b4c admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:18.453761 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37624fa5-bdaf-4f8e-9d2c-441529572b4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:18.456926 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37624fa5-bdaf-4f8e-9d2c-441529572b4c admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:18.701351 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 921/1846] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:41:18 2022] POST /identity/v3/projects => generated 383 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:41:18.707614 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bb26d3-8e6e-4964-95a8-e39a05a430b1 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:18.737281 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bb26d3-8e6e-4964-95a8-e39a05a430b1 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:18.863864 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92356da3-4c90-40f4-bcc0-7e964c98e7dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:18.864835 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92356da3-4c90-40f4-bcc0-7e964c98e7dc admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:18.865235 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92356da3-4c90-40f4-bcc0-7e964c98e7dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:18.865796 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92356da3-4c90-40f4-bcc0-7e964c98e7dc admin admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:18.867109 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92356da3-4c90-40f4-bcc0-7e964c98e7dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:18.870823 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92356da3-4c90-40f4-bcc0-7e964c98e7dc admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:19.056176 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 926/1847] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:41:18 2022] POST /identity/v3/users => generated 292 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:41:19.063166 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37624fa5-bdaf-4f8e-9d2c-441529572b4c admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:19.102257 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37624fa5-bdaf-4f8e-9d2c-441529572b4c admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:19.114494 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6072e7-fef3-452a-a001-c1d5e7ca1eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:19.117987 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6072e7-fef3-452a-a001-c1d5e7ca1eab admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:19.129169 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6072e7-fef3-452a-a001-c1d5e7ca1eab admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:19.129169 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6072e7-fef3-452a-a001-c1d5e7ca1eab admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:19.129169 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6072e7-fef3-452a-a001-c1d5e7ca1eab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:19.129169 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6072e7-fef3-452a-a001-c1d5e7ca1eab admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:19.148246 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 922/1848] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:41:19 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:41:19.153240 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92356da3-4c90-40f4-bcc0-7e964c98e7dc admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:19.188787 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92356da3-4c90-40f4-bcc0-7e964c98e7dc admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:19.208653 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90315716-f239-4591-b4ee-33ebac73d105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:19.209502 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90315716-f239-4591-b4ee-33ebac73d105 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:19.209885 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90315716-f239-4591-b4ee-33ebac73d105 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:19.210261 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90315716-f239-4591-b4ee-33ebac73d105 admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:19.211054 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90315716-f239-4591-b4ee-33ebac73d105 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:19.213202 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90315716-f239-4591-b4ee-33ebac73d105 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:19.231739 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 927/1849] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:41:19 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:41:19.236187 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6072e7-fef3-452a-a001-c1d5e7ca1eab admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:19.250284 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6072e7-fef3-452a-a001-c1d5e7ca1eab admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:19.257562 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440d40ba-b6b1-41d9-bc52-0f7f76bc56b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:19.259756 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440d40ba-b6b1-41d9-bc52-0f7f76bc56b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:19.260427 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440d40ba-b6b1-41d9-bc52-0f7f76bc56b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:19.261068 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440d40ba-b6b1-41d9-bc52-0f7f76bc56b4 admin admin] PATH_INFO: `/v3/projects/fef584f471254e6790f9f26f31b29543/users/d4ae2c4920d84bc98b176f88a6dd0f07/roles/e1f9b167c51c4a388cbc9594db8a465d` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:19.292909 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b388f65-3c63-436d-a25a-3ba3bc5751a6 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:19.293345 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b388f65-3c63-436d-a25a-3ba3bc5751a6 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:19.293674 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b388f65-3c63-436d-a25a-3ba3bc5751a6 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:19.295687 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 928/1850] 10.222.0.47 () {54 vars in 921 bytes} [Wed Aug 24 06:41:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:41:19.300483 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440d40ba-b6b1-41d9-bc52-0f7f76bc56b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fef584f471254e6790f9f26f31b29543, user_id=d4ae2c4920d84bc98b176f88a6dd0f07, role_id=e1f9b167c51c4a388cbc9594db8a465d)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:19.306794 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ced393-f036-46fa-af2c-b5799f4ab268 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:19.307550 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ced393-f036-46fa-af2c-b5799f4ab268 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:19.308054 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ced393-f036-46fa-af2c-b5799f4ab268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:19.309342 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440d40ba-b6b1-41d9-bc52-0f7f76bc56b4 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:19.352797 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-98ced393-f036-46fa-af2c-b5799f4ab268 None None] MFA Rules not processed for user `1f4f49c1b08c460f8ba6c451c9c92c2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:41:19.358619 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-98ced393-f036-46fa-af2c-b5799f4ab268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:19.431662 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 923/1851] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:41:19 2022] PUT /identity/v3/projects/fef584f471254e6790f9f26f31b29543/users/d4ae2c4920d84bc98b176f88a6dd0f07/roles/e1f9b167c51c4a388cbc9594db8a465d => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:41:19.443990 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d37418-163a-438b-bcf1-8ae034714e62 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:19.445067 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d37418-163a-438b-bcf1-8ae034714e62 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:19.445641 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d37418-163a-438b-bcf1-8ae034714e62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:19.477490 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 929/1852] 10.222.0.47 () {58 vars in 1018 bytes} [Wed Aug 24 06:41:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:41:19.483549 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-80d37418-163a-438b-bcf1-8ae034714e62 None None] MFA Rules not processed for user `d4ae2c4920d84bc98b176f88a6dd0f07`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:41:19.485165 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-80d37418-163a-438b-bcf1-8ae034714e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:19.505462 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ced393-f036-46fa-af2c-b5799f4ab268 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:19.508548 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-98ced393-f036-46fa-af2c-b5799f4ab268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:19.533400 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ced393-f036-46fa-af2c-b5799f4ab268 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:19.631357 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 924/1853] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:41:19 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:41:19.644750 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d37418-163a-438b-bcf1-8ae034714e62 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:19.645935 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18965a0-e4dc-4533-8fed-fd47076edb95 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4f49c1b08c460f8ba6c451c9c92c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:19.646511 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-80d37418-163a-438b-bcf1-8ae034714e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:19.646686 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18965a0-e4dc-4533-8fed-fd47076edb95 service nova] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:19.646987 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18965a0-e4dc-4533-8fed-fd47076edb95 service nova] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:19.647317 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18965a0-e4dc-4533-8fed-fd47076edb95 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:19.658296 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18965a0-e4dc-4533-8fed-fd47076edb95 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:19.660573 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18965a0-e4dc-4533-8fed-fd47076edb95 service nova] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:19.677821 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 930/1854] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:41:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:41:19.733676 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d37418-163a-438b-bcf1-8ae034714e62 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:19.738615 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfd3671-5009-40c4-985d-87f1a7f2efc4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36639dce4f97412f89d71367f650b901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:19.739250 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd3671-5009-40c4-985d-87f1a7f2efc4 service cinder] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:19.739527 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd3671-5009-40c4-985d-87f1a7f2efc4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:19.739791 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd3671-5009-40c4-985d-87f1a7f2efc4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:19.753924 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfd3671-5009-40c4-985d-87f1a7f2efc4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:19.756691 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfd3671-5009-40c4-985d-87f1a7f2efc4 service cinder] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:19.774407 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 925/1855] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:41:19 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:41:20.383292 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:41:23.515765 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18965a0-e4dc-4533-8fed-fd47076edb95 service nova] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:23.518245 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-b18965a0-e4dc-4533-8fed-fd47076edb95 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:23.664287 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18965a0-e4dc-4533-8fed-fd47076edb95 service nova] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:23.680770 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3317c26b-52dc-434a-8860-0388f5c2bf07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:23.682241 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3317c26b-52dc-434a-8860-0388f5c2bf07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:23.683297 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3317c26b-52dc-434a-8860-0388f5c2bf07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:23.684179 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3317c26b-52dc-434a-8860-0388f5c2bf07 admin admin] PATH_INFO: `/v3/users/6027122b2a8040d58e5bf9fc8a46400d` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:23.692064 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3317c26b-52dc-434a-8860-0388f5c2bf07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6027122b2a8040d58e5bf9fc8a46400d)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:23.709582 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3317c26b-52dc-434a-8860-0388f5c2bf07 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:24.055607 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:41:24.111568 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-3317c26b-52dc-434a-8860-0388f5c2bf07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6027122b2a8040d58e5bf9fc8a46400d'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:24.112760 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-3317c26b-52dc-434a-8860-0388f5c2bf07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6027122b2a8040d58e5bf9fc8a46400d'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:24.113424 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-3317c26b-52dc-434a-8860-0388f5c2bf07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6027122b2a8040d58e5bf9fc8a46400d'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:24.223773 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-3317c26b-52dc-434a-8860-0388f5c2bf07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6027122b2a8040d58e5bf9fc8a46400d'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:24.251686 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-3317c26b-52dc-434a-8860-0388f5c2bf07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6027122b2a8040d58e5bf9fc8a46400d'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:24.279853 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 931/1856] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:41:23 2022] DELETE /identity/v3/users/6027122b2a8040d58e5bf9fc8a46400d => generated 0 bytes in 765 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:41:24.287573 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de6e0d9-cb50-4d6d-9221-eb1d8fd7b87a None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:24.289325 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de6e0d9-cb50-4d6d-9221-eb1d8fd7b87a None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:24.290038 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de6e0d9-cb50-4d6d-9221-eb1d8fd7b87a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:24.350555 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-4de6e0d9-cb50-4d6d-9221-eb1d8fd7b87a None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:41:24.354569 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-4de6e0d9-cb50-4d6d-9221-eb1d8fd7b87a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:24.464103 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 926/1857] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:41:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:41:24.485033 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3317c26b-52dc-434a-8860-0388f5c2bf07 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:24.486495 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-3317c26b-52dc-434a-8860-0388f5c2bf07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:24.568114 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3317c26b-52dc-434a-8860-0388f5c2bf07 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:24.572452 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae1fbb4-9441-429e-8e70-c2adf0c7096e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:24.573080 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae1fbb4-9441-429e-8e70-c2adf0c7096e service neutron] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:24.573445 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae1fbb4-9441-429e-8e70-c2adf0c7096e service neutron] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:24.573769 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae1fbb4-9441-429e-8e70-c2adf0c7096e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:24.593583 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae1fbb4-9441-429e-8e70-c2adf0c7096e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:24.595294 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae1fbb4-9441-429e-8e70-c2adf0c7096e service neutron] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:24.610109 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 932/1858] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:41:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:41:25.704116 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de6e0d9-cb50-4d6d-9221-eb1d8fd7b87a None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:25.706424 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-4de6e0d9-cb50-4d6d-9221-eb1d8fd7b87a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:25.722283 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de6e0d9-cb50-4d6d-9221-eb1d8fd7b87a None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:25.727504 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59761541-694d-45e1-8887-69dcbe4f6493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:25.728144 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59761541-694d-45e1-8887-69dcbe4f6493 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:25.728495 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59761541-694d-45e1-8887-69dcbe4f6493 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:25.728778 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59761541-694d-45e1-8887-69dcbe4f6493 admin admin] PATH_INFO: `/v3/users/07828c08728d474cb4f4c6b21b90ebfb` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:25.729987 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59761541-694d-45e1-8887-69dcbe4f6493 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07828c08728d474cb4f4c6b21b90ebfb)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:25.732570 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59761541-694d-45e1-8887-69dcbe4f6493 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:25.743688 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae1fbb4-9441-429e-8e70-c2adf0c7096e service neutron] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:25.746343 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-dae1fbb4-9441-429e-8e70-c2adf0c7096e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:25.763780 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae1fbb4-9441-429e-8e70-c2adf0c7096e service neutron] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:25.769541 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b16b03-826e-470b-998a-184756dcfbf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:25.770689 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b16b03-826e-470b-998a-184756dcfbf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:25.771613 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b16b03-826e-470b-998a-184756dcfbf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:25.773390 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b16b03-826e-470b-998a-184756dcfbf3 admin admin] PATH_INFO: `/v3/projects/dbdecbd4f06542bea6308f575a89f0f9` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:25.774975 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b16b03-826e-470b-998a-184756dcfbf3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbdecbd4f06542bea6308f575a89f0f9)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:25.777020 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b16b03-826e-470b-998a-184756dcfbf3 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:26.188500 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:41:26.359346 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-a5b16b03-826e-470b-998a-184756dcfbf3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbdecbd4f06542bea6308f575a89f0f9', 'request_id': 'req-a5b16b03-826e-470b-998a-184756dcfbf3'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:26.364902 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-59761541-694d-45e1-8887-69dcbe4f6493 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07828c08728d474cb4f4c6b21b90ebfb'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:26.364902 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-59761541-694d-45e1-8887-69dcbe4f6493 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07828c08728d474cb4f4c6b21b90ebfb'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:26.364902 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-59761541-694d-45e1-8887-69dcbe4f6493 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07828c08728d474cb4f4c6b21b90ebfb'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:26.371668 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-59761541-694d-45e1-8887-69dcbe4f6493 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07828c08728d474cb4f4c6b21b90ebfb'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:26.595511 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-a5b16b03-826e-470b-998a-184756dcfbf3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbdecbd4f06542bea6308f575a89f0f9', 'request_id': 'req-a5b16b03-826e-470b-998a-184756dcfbf3'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:26.601326 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-a5b16b03-826e-470b-998a-184756dcfbf3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbdecbd4f06542bea6308f575a89f0f9', 'request_id': 'req-a5b16b03-826e-470b-998a-184756dcfbf3'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:26.620134 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-a5b16b03-826e-470b-998a-184756dcfbf3 admin admin] The token cache is being invalidate because project dbdecbd4f06542bea6308f575a89f0f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81723) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:41:26.620585 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-a5b16b03-826e-470b-998a-184756dcfbf3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:26.624600 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-a5b16b03-826e-470b-998a-184756dcfbf3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:26.628009 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 933/1859] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:41:25 2022] DELETE /identity/v3/projects/dbdecbd4f06542bea6308f575a89f0f9 => generated 0 bytes in 884 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:41:26.631361 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b16b03-826e-470b-998a-184756dcfbf3 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:26.632816 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-a5b16b03-826e-470b-998a-184756dcfbf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:26.705655 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b16b03-826e-470b-998a-184756dcfbf3 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:26.710829 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e61d10-44b1-40fe-a957-2f285dda14df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:26.711457 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e61d10-44b1-40fe-a957-2f285dda14df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:26.711735 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e61d10-44b1-40fe-a957-2f285dda14df admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:26.712004 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e61d10-44b1-40fe-a957-2f285dda14df admin admin] PATH_INFO: `/v3/users/7a034871ca5749769822e57384b2429d` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:26.713166 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e61d10-44b1-40fe-a957-2f285dda14df admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a034871ca5749769822e57384b2429d)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:26.715156 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e61d10-44b1-40fe-a957-2f285dda14df admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:26.833988 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-59761541-694d-45e1-8887-69dcbe4f6493 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07828c08728d474cb4f4c6b21b90ebfb'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:26.853091 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 927/1860] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:41:25 2022] DELETE /identity/v3/users/07828c08728d474cb4f4c6b21b90ebfb => generated 0 bytes in 1150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:41:26.858353 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd5e771-f961-45e4-b4c0-e611305b69d5 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:26.858707 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd5e771-f961-45e4-b4c0-e611305b69d5 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:26.858977 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd5e771-f961-45e4-b4c0-e611305b69d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:26.884763 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-dbd5e771-f961-45e4-b4c0-e611305b69d5 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:41:26.886909 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-dbd5e771-f961-45e4-b4c0-e611305b69d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:26.954669 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 928/1861] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:41:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:41:26.964736 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd5e771-f961-45e4-b4c0-e611305b69d5 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:26.966899 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-dbd5e771-f961-45e4-b4c0-e611305b69d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:27.031387 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd5e771-f961-45e4-b4c0-e611305b69d5 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:27.037344 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435169b7-eca8-4d4d-a259-6835c2d7593f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:27.037344 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435169b7-eca8-4d4d-a259-6835c2d7593f service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:27.037344 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435169b7-eca8-4d4d-a259-6835c2d7593f service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:27.037344 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435169b7-eca8-4d4d-a259-6835c2d7593f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:27.049027 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435169b7-eca8-4d4d-a259-6835c2d7593f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:27.051633 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435169b7-eca8-4d4d-a259-6835c2d7593f service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:27.064624 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 929/1862] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:41:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:41:27.267241 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-05e61d10-44b1-40fe-a957-2f285dda14df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a034871ca5749769822e57384b2429d'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:27.268037 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-05e61d10-44b1-40fe-a957-2f285dda14df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a034871ca5749769822e57384b2429d'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:27.268537 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-05e61d10-44b1-40fe-a957-2f285dda14df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a034871ca5749769822e57384b2429d'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:27.474561 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-05e61d10-44b1-40fe-a957-2f285dda14df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a034871ca5749769822e57384b2429d'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:27.502141 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-05e61d10-44b1-40fe-a957-2f285dda14df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a034871ca5749769822e57384b2429d'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:27.537815 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 934/1863] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:41:26 2022] DELETE /identity/v3/users/7a034871ca5749769822e57384b2429d => generated 0 bytes in 895 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:41:27.804254 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435169b7-eca8-4d4d-a259-6835c2d7593f service neutron] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:27.805771 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-435169b7-eca8-4d4d-a259-6835c2d7593f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:27.826836 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435169b7-eca8-4d4d-a259-6835c2d7593f service neutron] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:27.931957 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab546f5-ae85-40ba-8423-f5ac78f6551e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:27.932742 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab546f5-ae85-40ba-8423-f5ac78f6551e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:27.932854 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab546f5-ae85-40ba-8423-f5ac78f6551e admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:27.933059 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab546f5-ae85-40ba-8423-f5ac78f6551e admin admin] PATH_INFO: `/v3/projects/7158a44909f1494b9f4259b74fc786f9` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:27.934317 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab546f5-ae85-40ba-8423-f5ac78f6551e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7158a44909f1494b9f4259b74fc786f9)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:27.936860 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab546f5-ae85-40ba-8423-f5ac78f6551e admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:28.201123 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-6ab546f5-ae85-40ba-8423-f5ac78f6551e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7158a44909f1494b9f4259b74fc786f9', 'request_id': 'req-6ab546f5-ae85-40ba-8423-f5ac78f6551e'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:28.207322 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-6ab546f5-ae85-40ba-8423-f5ac78f6551e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7158a44909f1494b9f4259b74fc786f9', 'request_id': 'req-6ab546f5-ae85-40ba-8423-f5ac78f6551e'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:28.283202 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-6ab546f5-ae85-40ba-8423-f5ac78f6551e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7158a44909f1494b9f4259b74fc786f9', 'request_id': 'req-6ab546f5-ae85-40ba-8423-f5ac78f6551e'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:28.299935 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-6ab546f5-ae85-40ba-8423-f5ac78f6551e admin admin] The token cache is being invalidate because project 7158a44909f1494b9f4259b74fc786f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:41:28.300173 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-6ab546f5-ae85-40ba-8423-f5ac78f6551e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:28.300783 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-6ab546f5-ae85-40ba-8423-f5ac78f6551e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:28.305535 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 930/1864] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:41:27 2022] DELETE /identity/v3/projects/7158a44909f1494b9f4259b74fc786f9 => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:41:28.307035 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e61d10-44b1-40fe-a957-2f285dda14df admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:28.309068 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-05e61d10-44b1-40fe-a957-2f285dda14df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:28.330052 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e61d10-44b1-40fe-a957-2f285dda14df admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:28.420629 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d2a73b-2398-4469-b097-19f155daf1b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:28.421664 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d2a73b-2398-4469-b097-19f155daf1b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:28.422028 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d2a73b-2398-4469-b097-19f155daf1b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:28.422372 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d2a73b-2398-4469-b097-19f155daf1b4 admin admin] PATH_INFO: `/v3/users/27503206d9b84c69a74457fa777a17fc` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:28.423742 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d2a73b-2398-4469-b097-19f155daf1b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27503206d9b84c69a74457fa777a17fc)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:28.429560 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d2a73b-2398-4469-b097-19f155daf1b4 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:28.495067 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab546f5-ae85-40ba-8423-f5ac78f6551e admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:28.497893 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-6ab546f5-ae85-40ba-8423-f5ac78f6551e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:28.514167 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab546f5-ae85-40ba-8423-f5ac78f6551e admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:28.522681 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5439c5ae-391a-48b4-b21c-e2573bd96507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:28.522681 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5439c5ae-391a-48b4-b21c-e2573bd96507 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:28.522681 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5439c5ae-391a-48b4-b21c-e2573bd96507 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:28.522681 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5439c5ae-391a-48b4-b21c-e2573bd96507 admin admin] PATH_INFO: `/v3/projects/18e22cf7cd3b468eab2928fc244490af` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:28.526297 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5439c5ae-391a-48b4-b21c-e2573bd96507 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18e22cf7cd3b468eab2928fc244490af)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:28.530992 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5439c5ae-391a-48b4-b21c-e2573bd96507 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:28.697514 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-90d2a73b-2398-4469-b097-19f155daf1b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27503206d9b84c69a74457fa777a17fc'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:28.698274 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-90d2a73b-2398-4469-b097-19f155daf1b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27503206d9b84c69a74457fa777a17fc'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:28.698883 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-90d2a73b-2398-4469-b097-19f155daf1b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27503206d9b84c69a74457fa777a17fc'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:28.742479 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-5439c5ae-391a-48b4-b21c-e2573bd96507 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18e22cf7cd3b468eab2928fc244490af', 'request_id': 'req-5439c5ae-391a-48b4-b21c-e2573bd96507'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:28.772589 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-5439c5ae-391a-48b4-b21c-e2573bd96507 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18e22cf7cd3b468eab2928fc244490af', 'request_id': 'req-5439c5ae-391a-48b4-b21c-e2573bd96507'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:28.781715 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-90d2a73b-2398-4469-b097-19f155daf1b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27503206d9b84c69a74457fa777a17fc'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:28.806241 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-90d2a73b-2398-4469-b097-19f155daf1b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27503206d9b84c69a74457fa777a17fc'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:28.827851 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 935/1865] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:41:28 2022] DELETE /identity/v3/users/27503206d9b84c69a74457fa777a17fc => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:41:28.906540 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-5439c5ae-391a-48b4-b21c-e2573bd96507 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18e22cf7cd3b468eab2928fc244490af', 'request_id': 'req-5439c5ae-391a-48b4-b21c-e2573bd96507'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:28.939246 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-5439c5ae-391a-48b4-b21c-e2573bd96507 admin admin] The token cache is being invalidate because project 18e22cf7cd3b468eab2928fc244490af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:41:28.940083 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:41:28.948664 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-5439c5ae-391a-48b4-b21c-e2573bd96507 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:28.948664 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-5439c5ae-391a-48b4-b21c-e2573bd96507 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:28.948664 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 931/1866] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:41:28 2022] DELETE /identity/v3/projects/18e22cf7cd3b468eab2928fc244490af => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:41:28.953081 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec52645b-8098-4aaf-8192-8cb8a3a7db82 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:28.953566 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec52645b-8098-4aaf-8192-8cb8a3a7db82 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:28.954023 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec52645b-8098-4aaf-8192-8cb8a3a7db82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:28.992969 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-ec52645b-8098-4aaf-8192-8cb8a3a7db82 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:41:28.994244 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-ec52645b-8098-4aaf-8192-8cb8a3a7db82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:29.068277 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 936/1867] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:41:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:41:29.078687 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec32159e-da29-4160-b65d-0a47f53026db None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:29.079097 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec32159e-da29-4160-b65d-0a47f53026db None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:29.079481 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec32159e-da29-4160-b65d-0a47f53026db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:29.102664 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-ec32159e-da29-4160-b65d-0a47f53026db None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:41:29.103937 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-ec32159e-da29-4160-b65d-0a47f53026db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:29.112813 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 932/1868] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:41:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:41:29.117215 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec52645b-8098-4aaf-8192-8cb8a3a7db82 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:29.136361 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec52645b-8098-4aaf-8192-8cb8a3a7db82 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:29.142294 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e01bc6f-26af-4e9b-a30e-5bb548417b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:29.142896 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e01bc6f-26af-4e9b-a30e-5bb548417b8e admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:29.143296 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e01bc6f-26af-4e9b-a30e-5bb548417b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:29.143451 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e01bc6f-26af-4e9b-a30e-5bb548417b8e admin admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:29.144142 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e01bc6f-26af-4e9b-a30e-5bb548417b8e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:41:29.144449 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e01bc6f-26af-4e9b-a30e-5bb548417b8e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:29.146089 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e01bc6f-26af-4e9b-a30e-5bb548417b8e admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:29.159438 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 937/1869] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 24 06:41:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:41:29.164495 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec32159e-da29-4160-b65d-0a47f53026db None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:29.175365 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec32159e-da29-4160-b65d-0a47f53026db None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:29.180141 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c66112-04f4-4160-9d3c-df5817722ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:29.180617 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c66112-04f4-4160-9d3c-df5817722ba5 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:29.180894 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c66112-04f4-4160-9d3c-df5817722ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:29.181172 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c66112-04f4-4160-9d3c-df5817722ba5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:29.181863 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c66112-04f4-4160-9d3c-df5817722ba5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:29.183880 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c66112-04f4-4160-9d3c-df5817722ba5 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:29.293148 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 933/1870] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:41:29 2022] POST /identity/v3/projects => generated 385 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:41:29.298282 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e01bc6f-26af-4e9b-a30e-5bb548417b8e admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:29.309559 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e01bc6f-26af-4e9b-a30e-5bb548417b8e admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:29.384068 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41052a7b-24dd-4fa8-a9ff-ddd3ca963cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:29.385757 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41052a7b-24dd-4fa8-a9ff-ddd3ca963cf7 admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:29.386234 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41052a7b-24dd-4fa8-a9ff-ddd3ca963cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:29.386522 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41052a7b-24dd-4fa8-a9ff-ddd3ca963cf7 admin admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:29.388998 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41052a7b-24dd-4fa8-a9ff-ddd3ca963cf7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:29.391270 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41052a7b-24dd-4fa8-a9ff-ddd3ca963cf7 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:29.497669 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c66112-04f4-4160-9d3c-df5817722ba5 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:29.499530 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-45c66112-04f4-4160-9d3c-df5817722ba5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:29.519894 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:41:29.521782 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 938/1871] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:41:29 2022] POST /identity/v3/users => generated 293 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:41:29.527307 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41052a7b-24dd-4fa8-a9ff-ddd3ca963cf7 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:29.537014 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c66112-04f4-4160-9d3c-df5817722ba5 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:29.547641 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52fb54f-5f4d-4af5-9a47-87715d471f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:29.548301 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52fb54f-5f4d-4af5-9a47-87715d471f28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:29.548587 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52fb54f-5f4d-4af5-9a47-87715d471f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:29.548860 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52fb54f-5f4d-4af5-9a47-87715d471f28 admin admin] PATH_INFO: `/v3/projects/0a27f1fc29fc4e0eb953a5aee22f5538` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:29.550239 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52fb54f-5f4d-4af5-9a47-87715d471f28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a27f1fc29fc4e0eb953a5aee22f5538)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:29.556558 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41052a7b-24dd-4fa8-a9ff-ddd3ca963cf7 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:29.556701 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52fb54f-5f4d-4af5-9a47-87715d471f28 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:29.558268 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365cbb14-4636-459f-bd6c-2fa6b1ce7e3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:29.558683 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365cbb14-4636-459f-bd6c-2fa6b1ce7e3c admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:29.560511 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365cbb14-4636-459f-bd6c-2fa6b1ce7e3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:29.561116 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365cbb14-4636-459f-bd6c-2fa6b1ce7e3c admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:29.562028 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365cbb14-4636-459f-bd6c-2fa6b1ce7e3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:29.563868 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365cbb14-4636-459f-bd6c-2fa6b1ce7e3c admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:29.582315 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 939/1872] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:41:29 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:41:29.586316 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365cbb14-4636-459f-bd6c-2fa6b1ce7e3c admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:29.603179 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365cbb14-4636-459f-bd6c-2fa6b1ce7e3c admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:29.614065 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26961f93-beb1-45bb-a960-0dfad12ae0b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:29.614065 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26961f93-beb1-45bb-a960-0dfad12ae0b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:29.614065 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26961f93-beb1-45bb-a960-0dfad12ae0b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:29.614065 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26961f93-beb1-45bb-a960-0dfad12ae0b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:29.614409 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26961f93-beb1-45bb-a960-0dfad12ae0b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:29.614409 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26961f93-beb1-45bb-a960-0dfad12ae0b1 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:29.635514 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 940/1873] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:41:29 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:41:29.637633 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26961f93-beb1-45bb-a960-0dfad12ae0b1 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:29.648752 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26961f93-beb1-45bb-a960-0dfad12ae0b1 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:29.654314 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14bf08d-d254-4412-a2c2-2127e4e35947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:29.654992 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14bf08d-d254-4412-a2c2-2127e4e35947 admin admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:29.655443 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14bf08d-d254-4412-a2c2-2127e4e35947 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:29.655884 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14bf08d-d254-4412-a2c2-2127e4e35947 admin admin] PATH_INFO: `/v3/projects/8c9f8c2822bc40829857bf827b5eddb1/users/abde191473584a778b14bd77a06aa835/roles/e1f9b167c51c4a388cbc9594db8a465d` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:29.680928 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14bf08d-d254-4412-a2c2-2127e4e35947 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c9f8c2822bc40829857bf827b5eddb1, user_id=abde191473584a778b14bd77a06aa835, role_id=e1f9b167c51c4a388cbc9594db8a465d)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:29.682086 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14bf08d-d254-4412-a2c2-2127e4e35947 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:29.812119 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 941/1874] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:41:29 2022] PUT /identity/v3/projects/8c9f8c2822bc40829857bf827b5eddb1/users/abde191473584a778b14bd77a06aa835/roles/e1f9b167c51c4a388cbc9594db8a465d => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:41:29.830931 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806097e2-512e-4ed3-ae06-80fd46421216 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:29.831789 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806097e2-512e-4ed3-ae06-80fd46421216 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:29.832293 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806097e2-512e-4ed3-ae06-80fd46421216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:29.834808 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-f52fb54f-5f4d-4af5-9a47-87715d471f28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a27f1fc29fc4e0eb953a5aee22f5538', 'request_id': 'req-f52fb54f-5f4d-4af5-9a47-87715d471f28'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:29.841766 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-f52fb54f-5f4d-4af5-9a47-87715d471f28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a27f1fc29fc4e0eb953a5aee22f5538', 'request_id': 'req-f52fb54f-5f4d-4af5-9a47-87715d471f28'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:29.874784 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-806097e2-512e-4ed3-ae06-80fd46421216 None None] MFA Rules not processed for user `abde191473584a778b14bd77a06aa835`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:41:29.876088 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-806097e2-512e-4ed3-ae06-80fd46421216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:29.933901 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-f52fb54f-5f4d-4af5-9a47-87715d471f28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a27f1fc29fc4e0eb953a5aee22f5538', 'request_id': 'req-f52fb54f-5f4d-4af5-9a47-87715d471f28'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:29.947043 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-f52fb54f-5f4d-4af5-9a47-87715d471f28 admin admin] The token cache is being invalidate because project 0a27f1fc29fc4e0eb953a5aee22f5538 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:41:29.947043 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-f52fb54f-5f4d-4af5-9a47-87715d471f28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:29.951377 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-f52fb54f-5f4d-4af5-9a47-87715d471f28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:29.951377 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 934/1875] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:41:29 2022] DELETE /identity/v3/projects/0a27f1fc29fc4e0eb953a5aee22f5538 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:41:29.952074 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 942/1876] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:41:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:41:29.954731 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffa379e-227a-4690-a05b-5218dfaa9b29 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:29.955591 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffa379e-227a-4690-a05b-5218dfaa9b29 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:29.955591 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffa379e-227a-4690-a05b-5218dfaa9b29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:29.955947 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806097e2-512e-4ed3-ae06-80fd46421216 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:29.987155 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-9ffa379e-227a-4690-a05b-5218dfaa9b29 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:41:29.988405 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-9ffa379e-227a-4690-a05b-5218dfaa9b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:30.031660 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806097e2-512e-4ed3-ae06-80fd46421216 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:30.039766 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee33458d-e003-4135-abf3-b7993a7e3264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:30.040558 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee33458d-e003-4135-abf3-b7993a7e3264 admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:30.040902 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee33458d-e003-4135-abf3-b7993a7e3264 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:30.041370 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee33458d-e003-4135-abf3-b7993a7e3264 admin admin] PATH_INFO: `/v3/projects` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:30.042229 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee33458d-e003-4135-abf3-b7993a7e3264 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:30.052948 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee33458d-e003-4135-abf3-b7993a7e3264 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:30.068365 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 935/1877] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:41:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:41:30.077211 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45d840f-c0b6-4eff-adb9-3ac9892d5b04 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:30.077502 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45d840f-c0b6-4eff-adb9-3ac9892d5b04 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:30.077787 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45d840f-c0b6-4eff-adb9-3ac9892d5b04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:30.098999 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-f45d840f-c0b6-4eff-adb9-3ac9892d5b04 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:41:30.100553 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-f45d840f-c0b6-4eff-adb9-3ac9892d5b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:30.110656 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 936/1878] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:41:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:41:30.115277 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45d840f-c0b6-4eff-adb9-3ac9892d5b04 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:30.130797 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45d840f-c0b6-4eff-adb9-3ac9892d5b04 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:30.134200 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716a9d30-8bdb-4c1b-b6ca-b959689d1743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:30.134852 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716a9d30-8bdb-4c1b-b6ca-b959689d1743 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:30.135130 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716a9d30-8bdb-4c1b-b6ca-b959689d1743 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:30.135413 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716a9d30-8bdb-4c1b-b6ca-b959689d1743 admin admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:30.136184 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716a9d30-8bdb-4c1b-b6ca-b959689d1743 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:41:30.136420 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716a9d30-8bdb-4c1b-b6ca-b959689d1743 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:30.138774 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716a9d30-8bdb-4c1b-b6ca-b959689d1743 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:30.165668 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 937/1879] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 24 06:41:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:41:30.170599 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716a9d30-8bdb-4c1b-b6ca-b959689d1743 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:30.182331 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716a9d30-8bdb-4c1b-b6ca-b959689d1743 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:30.188664 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15930e44-cfd7-45c3-bf81-3e316ecdad86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:30.189588 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15930e44-cfd7-45c3-bf81-3e316ecdad86 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:30.190190 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15930e44-cfd7-45c3-bf81-3e316ecdad86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:30.192373 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15930e44-cfd7-45c3-bf81-3e316ecdad86 admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:30.193136 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15930e44-cfd7-45c3-bf81-3e316ecdad86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:30.205264 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15930e44-cfd7-45c3-bf81-3e316ecdad86 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:30.224091 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 943/1880] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:41:29 2022] POST /identity/v3/projects => generated 387 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:41:30.228307 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee33458d-e003-4135-abf3-b7993a7e3264 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:30.230106 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-ee33458d-e003-4135-abf3-b7993a7e3264 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:30.304646 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 938/1881] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:41:30 2022] POST /identity/v3/projects => generated 363 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:41:30.308822 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee33458d-e003-4135-abf3-b7993a7e3264 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:30.311955 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15930e44-cfd7-45c3-bf81-3e316ecdad86 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:30.404669 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654cd560-e085-480e-b56e-fb3314baf859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:30.405405 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15930e44-cfd7-45c3-bf81-3e316ecdad86 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:30.405752 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654cd560-e085-480e-b56e-fb3314baf859 admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:30.406173 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654cd560-e085-480e-b56e-fb3314baf859 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:30.406661 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654cd560-e085-480e-b56e-fb3314baf859 admin admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:30.407507 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654cd560-e085-480e-b56e-fb3314baf859 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:30.409487 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654cd560-e085-480e-b56e-fb3314baf859 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:30.411232 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c41fec7-71a1-4879-8efd-39e8fa4d3745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:30.412705 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c41fec7-71a1-4879-8efd-39e8fa4d3745 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:30.413165 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c41fec7-71a1-4879-8efd-39e8fa4d3745 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:30.413680 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c41fec7-71a1-4879-8efd-39e8fa4d3745 admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:30.414869 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c41fec7-71a1-4879-8efd-39e8fa4d3745 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:30.418167 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c41fec7-71a1-4879-8efd-39e8fa4d3745 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:30.510599 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 944/1882] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:41:30 2022] POST /identity/v3/users => generated 294 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:41:30.519435 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 939/1883] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:41:30 2022] POST /identity/v3/users => generated 282 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:41:30.526468 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654cd560-e085-480e-b56e-fb3314baf859 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:30.526878 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c41fec7-71a1-4879-8efd-39e8fa4d3745 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:30.543057 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654cd560-e085-480e-b56e-fb3314baf859 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:30.543751 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c41fec7-71a1-4879-8efd-39e8fa4d3745 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:30.552947 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4ab562-fce1-47ac-a8e6-bb02fc36cbe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:30.558275 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4ab562-fce1-47ac-a8e6-bb02fc36cbe6 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:30.558275 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4ab562-fce1-47ac-a8e6-bb02fc36cbe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:30.558490 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee22ade-c34c-4619-b7ab-4eacb5d1b744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:30.558614 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4ab562-fce1-47ac-a8e6-bb02fc36cbe6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:30.558710 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee22ade-c34c-4619-b7ab-4eacb5d1b744 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:30.558807 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4ab562-fce1-47ac-a8e6-bb02fc36cbe6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:30.558896 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee22ade-c34c-4619-b7ab-4eacb5d1b744 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:30.558896 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee22ade-c34c-4619-b7ab-4eacb5d1b744 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:30.558896 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee22ade-c34c-4619-b7ab-4eacb5d1b744 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:30.559692 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4ab562-fce1-47ac-a8e6-bb02fc36cbe6 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:30.559774 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee22ade-c34c-4619-b7ab-4eacb5d1b744 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:30.584383 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 940/1884] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:41:30 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:41:30.594961 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee22ade-c34c-4619-b7ab-4eacb5d1b744 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:30.596385 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 945/1885] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:41:30 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:41:30.601600 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4ab562-fce1-47ac-a8e6-bb02fc36cbe6 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:30.649166 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4ab562-fce1-47ac-a8e6-bb02fc36cbe6 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:30.651705 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee22ade-c34c-4619-b7ab-4eacb5d1b744 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:30.676498 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be010978-3418-4006-bdee-c8458ecb3e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:30.678201 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be010978-3418-4006-bdee-c8458ecb3e3b admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:30.679043 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be010978-3418-4006-bdee-c8458ecb3e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:30.680172 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be010978-3418-4006-bdee-c8458ecb3e3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:30.680950 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8094da2-c1de-4184-8ab5-1392be4c1c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:30.681819 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be010978-3418-4006-bdee-c8458ecb3e3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:30.684828 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8094da2-c1de-4184-8ab5-1392be4c1c3d admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:30.684828 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8094da2-c1de-4184-8ab5-1392be4c1c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:30.684828 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8094da2-c1de-4184-8ab5-1392be4c1c3d admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:30.684828 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8094da2-c1de-4184-8ab5-1392be4c1c3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:30.687672 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be010978-3418-4006-bdee-c8458ecb3e3b admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:30.690099 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8094da2-c1de-4184-8ab5-1392be4c1c3d admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:30.714714 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 946/1886] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:41:30 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:41:30.721374 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be010978-3418-4006-bdee-c8458ecb3e3b admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:30.724038 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 941/1887] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:41:30 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:41:30.731665 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8094da2-c1de-4184-8ab5-1392be4c1c3d admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:30.756294 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be010978-3418-4006-bdee-c8458ecb3e3b admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:30.775366 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8094da2-c1de-4184-8ab5-1392be4c1c3d admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:30.781430 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02b4f80-b0cb-4846-95e0-59a3e5dcf394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:30.782120 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02b4f80-b0cb-4846-95e0-59a3e5dcf394 admin admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:30.782403 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02b4f80-b0cb-4846-95e0-59a3e5dcf394 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:30.783345 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02b4f80-b0cb-4846-95e0-59a3e5dcf394 admin admin] PATH_INFO: `/v3/projects/4bf396234e3540198914d2bc13697ffa/users/446b6008e7ac4c118c47ec4f7b6dcc6d/roles/e1f9b167c51c4a388cbc9594db8a465d` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:30.790721 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfb3513-80ac-4f8c-9a76-8b82f08e4fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:30.791940 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfb3513-80ac-4f8c-9a76-8b82f08e4fac admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:30.794182 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfb3513-80ac-4f8c-9a76-8b82f08e4fac admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:30.794324 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfb3513-80ac-4f8c-9a76-8b82f08e4fac admin admin] PATH_INFO: `/v3/projects/87571c73f1b442b58d174fc3b6adc8b5/users/e5b9c1167f1f4a77b94c0e6843c52ef4/roles/e1f9b167c51c4a388cbc9594db8a465d` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:30.827940 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02b4f80-b0cb-4846-95e0-59a3e5dcf394 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bf396234e3540198914d2bc13697ffa, user_id=446b6008e7ac4c118c47ec4f7b6dcc6d, role_id=e1f9b167c51c4a388cbc9594db8a465d)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:30.832644 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02b4f80-b0cb-4846-95e0-59a3e5dcf394 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:30.843664 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfb3513-80ac-4f8c-9a76-8b82f08e4fac admin admin] RBAC: Authorizing `identity:create_grant(project_id=87571c73f1b442b58d174fc3b6adc8b5, user_id=e5b9c1167f1f4a77b94c0e6843c52ef4, role_id=e1f9b167c51c4a388cbc9594db8a465d)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:30.847764 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfb3513-80ac-4f8c-9a76-8b82f08e4fac admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:30.961714 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 942/1888] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:41:30 2022] PUT /identity/v3/projects/87571c73f1b442b58d174fc3b6adc8b5/users/e5b9c1167f1f4a77b94c0e6843c52ef4/roles/e1f9b167c51c4a388cbc9594db8a465d => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:41:30.962591 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 947/1889] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:41:30 2022] PUT /identity/v3/projects/4bf396234e3540198914d2bc13697ffa/users/446b6008e7ac4c118c47ec4f7b6dcc6d/roles/e1f9b167c51c4a388cbc9594db8a465d => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:41:30.975012 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fa1bdd-b3a6-419d-aea5-7294b5319e05 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:30.975558 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fa1bdd-b3a6-419d-aea5-7294b5319e05 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:30.975836 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fa1bdd-b3a6-419d-aea5-7294b5319e05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:30.980106 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e11e6f-fc6d-4dbf-ac79-1126d21a82f4 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:30.980395 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e11e6f-fc6d-4dbf-ac79-1126d21a82f4 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:30.980674 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e11e6f-fc6d-4dbf-ac79-1126d21a82f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:31.019008 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-21fa1bdd-b3a6-419d-aea5-7294b5319e05 None None] MFA Rules not processed for user `446b6008e7ac4c118c47ec4f7b6dcc6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:41:31.020315 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-21fa1bdd-b3a6-419d-aea5-7294b5319e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:31.038970 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-a1e11e6f-fc6d-4dbf-ac79-1126d21a82f4 None None] MFA Rules not processed for user `e5b9c1167f1f4a77b94c0e6843c52ef4`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:41:31.040493 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-a1e11e6f-fc6d-4dbf-ac79-1126d21a82f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:31.102150 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 948/1890] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:41:30 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:41:31.108064 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fa1bdd-b3a6-419d-aea5-7294b5319e05 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:31.158995 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 943/1891] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:41:30 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:41:31.163732 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e11e6f-fc6d-4dbf-ac79-1126d21a82f4 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:31.209516 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fa1bdd-b3a6-419d-aea5-7294b5319e05 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:31.215394 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f05bd9-0fc8-4f28-8bda-b33f287a927a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:31.216579 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f05bd9-0fc8-4f28-8bda-b33f287a927a admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:31.216876 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f05bd9-0fc8-4f28-8bda-b33f287a927a admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:31.217285 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f05bd9-0fc8-4f28-8bda-b33f287a927a admin admin] PATH_INFO: `/v3/projects` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:31.217943 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f05bd9-0fc8-4f28-8bda-b33f287a927a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:31.220934 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f05bd9-0fc8-4f28-8bda-b33f287a927a admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:31.258003 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:41:31.259533 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e11e6f-fc6d-4dbf-ac79-1126d21a82f4 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:31.275842 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e814f0-a916-43fd-a171-354d7f8af7f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:31.276699 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e814f0-a916-43fd-a171-354d7f8af7f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:31.277053 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e814f0-a916-43fd-a171-354d7f8af7f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:31.279308 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e814f0-a916-43fd-a171-354d7f8af7f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:31.280212 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e814f0-a916-43fd-a171-354d7f8af7f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:31.282522 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e814f0-a916-43fd-a171-354d7f8af7f5 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:31.363405 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 949/1892] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:41:31 2022] POST /identity/v3/projects => generated 385 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:41:31.364474 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f05bd9-0fc8-4f28-8bda-b33f287a927a admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:31.443500 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 944/1893] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:41:31 2022] POST /identity/v3/projects => generated 363 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:41:31.446898 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e814f0-a916-43fd-a171-354d7f8af7f5 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:31.456662 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f05bd9-0fc8-4f28-8bda-b33f287a927a admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:31.573972 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e814f0-a916-43fd-a171-354d7f8af7f5 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:31.577009 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad34de4-58b9-423b-9da0-c143b1604501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:31.580910 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059ef981-bfdb-44f7-a8f7-cf56f7f0afac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:31.581637 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059ef981-bfdb-44f7-a8f7-cf56f7f0afac admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:31.582121 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059ef981-bfdb-44f7-a8f7-cf56f7f0afac admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:31.582433 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059ef981-bfdb-44f7-a8f7-cf56f7f0afac admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:31.583061 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059ef981-bfdb-44f7-a8f7-cf56f7f0afac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:31.585697 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059ef981-bfdb-44f7-a8f7-cf56f7f0afac admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:31.586347 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad34de4-58b9-423b-9da0-c143b1604501 admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:31.586754 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad34de4-58b9-423b-9da0-c143b1604501 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:31.589629 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad34de4-58b9-423b-9da0-c143b1604501 admin admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:31.590524 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad34de4-58b9-423b-9da0-c143b1604501 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:31.594109 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad34de4-58b9-423b-9da0-c143b1604501 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:31.696090 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 945/1894] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:41:31 2022] POST /identity/v3/users => generated 288 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:41:31.700653 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059ef981-bfdb-44f7-a8f7-cf56f7f0afac admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:31.711145 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059ef981-bfdb-44f7-a8f7-cf56f7f0afac admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:31.716230 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31c1c9f-bf2f-4a0b-870f-c6239d95f9e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:31.717073 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c1c9f-bf2f-4a0b-870f-c6239d95f9e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:31.717452 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c1c9f-bf2f-4a0b-870f-c6239d95f9e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:31.717806 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c1c9f-bf2f-4a0b-870f-c6239d95f9e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:31.718455 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31c1c9f-bf2f-4a0b-870f-c6239d95f9e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:31.720783 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31c1c9f-bf2f-4a0b-870f-c6239d95f9e0 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:31.732084 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 946/1895] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:41:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:41:31.737639 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31c1c9f-bf2f-4a0b-870f-c6239d95f9e0 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:31.745297 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 950/1896] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:41:31 2022] POST /identity/v3/users => generated 299 bytes in 381 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:41:31.750463 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad34de4-58b9-423b-9da0-c143b1604501 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:31.752400 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31c1c9f-bf2f-4a0b-870f-c6239d95f9e0 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:31.758501 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe85c26c-1856-44c7-bb89-5da5bb1f2b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:31.758501 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe85c26c-1856-44c7-bb89-5da5bb1f2b3b admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:31.758939 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe85c26c-1856-44c7-bb89-5da5bb1f2b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:31.759223 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe85c26c-1856-44c7-bb89-5da5bb1f2b3b admin admin] PATH_INFO: `/v3/projects/f77f41d338954547bd204f0c65a0e9ef/users/94b65585ecf445d388d0c30a01ff8777/roles/486144ad1bcd417e86acdbc1d14327bf` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:31.763737 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad34de4-58b9-423b-9da0-c143b1604501 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:31.769303 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056e640d-9b0b-47bf-b616-b9ccc8c9f140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:31.770028 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056e640d-9b0b-47bf-b616-b9ccc8c9f140 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:31.770370 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056e640d-9b0b-47bf-b616-b9ccc8c9f140 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:31.770648 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056e640d-9b0b-47bf-b616-b9ccc8c9f140 admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:31.771297 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056e640d-9b0b-47bf-b616-b9ccc8c9f140 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:31.772916 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056e640d-9b0b-47bf-b616-b9ccc8c9f140 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:31.775653 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe85c26c-1856-44c7-bb89-5da5bb1f2b3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f77f41d338954547bd204f0c65a0e9ef, user_id=94b65585ecf445d388d0c30a01ff8777, role_id=486144ad1bcd417e86acdbc1d14327bf)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:31.778485 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe85c26c-1856-44c7-bb89-5da5bb1f2b3b admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:31.787866 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 951/1897] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:41:31 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:41:31.789410 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056e640d-9b0b-47bf-b616-b9ccc8c9f140 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:31.800920 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056e640d-9b0b-47bf-b616-b9ccc8c9f140 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:31.806031 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809b82f5-9233-4501-80c0-ae31239a1975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:31.806851 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809b82f5-9233-4501-80c0-ae31239a1975 admin admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:31.807131 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809b82f5-9233-4501-80c0-ae31239a1975 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:31.807407 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809b82f5-9233-4501-80c0-ae31239a1975 admin admin] PATH_INFO: `/v3/projects/fbe7372b867e475bb787f41afd2e3b49/users/eacae54b966944e28c112fe0efbdbc65/roles/486144ad1bcd417e86acdbc1d14327bf` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:31.824388 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809b82f5-9233-4501-80c0-ae31239a1975 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbe7372b867e475bb787f41afd2e3b49, user_id=eacae54b966944e28c112fe0efbdbc65, role_id=486144ad1bcd417e86acdbc1d14327bf)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:31.827306 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809b82f5-9233-4501-80c0-ae31239a1975 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:31.839663 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:41:31.957005 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 947/1898] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:41:31 2022] PUT /identity/v3/projects/f77f41d338954547bd204f0c65a0e9ef/users/94b65585ecf445d388d0c30a01ff8777/roles/486144ad1bcd417e86acdbc1d14327bf => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:41:31.958257 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe85c26c-1856-44c7-bb89-5da5bb1f2b3b admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:31.960763 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-fe85c26c-1856-44c7-bb89-5da5bb1f2b3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:31.995398 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 952/1899] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:41:31 2022] PUT /identity/v3/projects/fbe7372b867e475bb787f41afd2e3b49/users/eacae54b966944e28c112fe0efbdbc65/roles/486144ad1bcd417e86acdbc1d14327bf => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:41:31.995634 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809b82f5-9233-4501-80c0-ae31239a1975 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:32.086465 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe85c26c-1856-44c7-bb89-5da5bb1f2b3b admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:32.124889 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809b82f5-9233-4501-80c0-ae31239a1975 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:32.131946 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bd0de5-8deb-4008-b413-00c14a5b0be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:32.132907 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bd0de5-8deb-4008-b413-00c14a5b0be2 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:32.133348 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bd0de5-8deb-4008-b413-00c14a5b0be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:32.133812 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bd0de5-8deb-4008-b413-00c14a5b0be2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:32.134488 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bd0de5-8deb-4008-b413-00c14a5b0be2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:32.136550 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bd0de5-8deb-4008-b413-00c14a5b0be2 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:32.148132 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 953/1900] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:41:31 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:41:32.148841 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bd0de5-8deb-4008-b413-00c14a5b0be2 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:32.160534 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bd0de5-8deb-4008-b413-00c14a5b0be2 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:32.178277 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2e68ef-172a-4cde-99d8-d69ade9842f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:32.178277 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2e68ef-172a-4cde-99d8-d69ade9842f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:32.178277 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2e68ef-172a-4cde-99d8-d69ade9842f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:32.180553 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2e68ef-172a-4cde-99d8-d69ade9842f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:32.180553 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2e68ef-172a-4cde-99d8-d69ade9842f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:32.180732 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2e68ef-172a-4cde-99d8-d69ade9842f2 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:32.195666 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa75ff87-f12d-47a3-a2ca-ac6fd81eb68f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:32.196498 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa75ff87-f12d-47a3-a2ca-ac6fd81eb68f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:32.196789 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa75ff87-f12d-47a3-a2ca-ac6fd81eb68f admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:32.197124 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa75ff87-f12d-47a3-a2ca-ac6fd81eb68f admin admin] PATH_INFO: `/v3/users/b5abf796f042439bb93751971206a963` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:32.198404 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa75ff87-f12d-47a3-a2ca-ac6fd81eb68f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5abf796f042439bb93751971206a963)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:32.203342 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa75ff87-f12d-47a3-a2ca-ac6fd81eb68f admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:32.221333 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:41:32.222473 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 954/1901] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:41:32 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:41:32.223834 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2e68ef-172a-4cde-99d8-d69ade9842f2 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:32.265277 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2e68ef-172a-4cde-99d8-d69ade9842f2 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:32.284775 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f137cc3-dd88-4415-b625-01a55e650905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:32.284775 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f137cc3-dd88-4415-b625-01a55e650905 admin admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:32.284775 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f137cc3-dd88-4415-b625-01a55e650905 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:32.284775 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f137cc3-dd88-4415-b625-01a55e650905 admin admin] PATH_INFO: `/v3/projects/f77f41d338954547bd204f0c65a0e9ef/users/94b65585ecf445d388d0c30a01ff8777/roles/486144ad1bcd417e86acdbc1d14327bf` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:32.294228 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f137cc3-dd88-4415-b625-01a55e650905 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f77f41d338954547bd204f0c65a0e9ef, user_id=94b65585ecf445d388d0c30a01ff8777, role_id=486144ad1bcd417e86acdbc1d14327bf)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:32.296995 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f137cc3-dd88-4415-b625-01a55e650905 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:32.311570 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 955/1902] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:41:32 2022] PUT /identity/v3/projects/f77f41d338954547bd204f0c65a0e9ef/users/94b65585ecf445d388d0c30a01ff8777/roles/486144ad1bcd417e86acdbc1d14327bf => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:41:32.312479 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f137cc3-dd88-4415-b625-01a55e650905 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:32.408587 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f137cc3-dd88-4415-b625-01a55e650905 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:32.414208 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a8abfe-27e4-4079-9642-d7a7bcde8198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:32.414896 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a8abfe-27e4-4079-9642-d7a7bcde8198 admin admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:32.415178 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a8abfe-27e4-4079-9642-d7a7bcde8198 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:32.415452 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a8abfe-27e4-4079-9642-d7a7bcde8198 admin admin] PATH_INFO: `/v3/projects/fbe7372b867e475bb787f41afd2e3b49/users/eacae54b966944e28c112fe0efbdbc65/roles/486144ad1bcd417e86acdbc1d14327bf` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:32.417644 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a8abfe-27e4-4079-9642-d7a7bcde8198 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbe7372b867e475bb787f41afd2e3b49, user_id=eacae54b966944e28c112fe0efbdbc65, role_id=486144ad1bcd417e86acdbc1d14327bf)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:32.422228 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a8abfe-27e4-4079-9642-d7a7bcde8198 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:32.436319 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 956/1903] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:41:32 2022] PUT /identity/v3/projects/fbe7372b867e475bb787f41afd2e3b49/users/eacae54b966944e28c112fe0efbdbc65/roles/486144ad1bcd417e86acdbc1d14327bf => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:41:32.445599 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0ea443-e492-42fc-85f2-aad90fbfc961 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:32.445599 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0ea443-e492-42fc-85f2-aad90fbfc961 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:32.445599 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0ea443-e492-42fc-85f2-aad90fbfc961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:32.474097 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-dc0ea443-e492-42fc-85f2-aad90fbfc961 None None] MFA Rules not processed for user `94b65585ecf445d388d0c30a01ff8777`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:41:32.475610 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-dc0ea443-e492-42fc-85f2-aad90fbfc961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:32.532853 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-fa75ff87-f12d-47a3-a2ca-ac6fd81eb68f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5abf796f042439bb93751971206a963'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:32.533900 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-fa75ff87-f12d-47a3-a2ca-ac6fd81eb68f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5abf796f042439bb93751971206a963'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:32.535096 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-fa75ff87-f12d-47a3-a2ca-ac6fd81eb68f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5abf796f042439bb93751971206a963'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:32.545208 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-fa75ff87-f12d-47a3-a2ca-ac6fd81eb68f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5abf796f042439bb93751971206a963'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:32.554790 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 957/1904] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:41:32 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:41:32.555909 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e0e185-29d6-4eae-8eb3-17e59b8348b3 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:32.556487 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e0e185-29d6-4eae-8eb3-17e59b8348b3 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:32.556595 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e0e185-29d6-4eae-8eb3-17e59b8348b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:32.600442 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-b1e0e185-29d6-4eae-8eb3-17e59b8348b3 None None] MFA Rules not processed for user `eacae54b966944e28c112fe0efbdbc65`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:41:32.601821 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-b1e0e185-29d6-4eae-8eb3-17e59b8348b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:32.660243 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-fa75ff87-f12d-47a3-a2ca-ac6fd81eb68f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5abf796f042439bb93751971206a963'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:32.687561 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 948/1905] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:41:31 2022] DELETE /identity/v3/users/b5abf796f042439bb93751971206a963 => generated 0 bytes in 730 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:41:32.689287 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa75ff87-f12d-47a3-a2ca-ac6fd81eb68f admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:32.692100 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-fa75ff87-f12d-47a3-a2ca-ac6fd81eb68f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:32.767116 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 958/1906] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:41:32 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:41:32.779818 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c396be-0034-44fa-876f-5ad456e1784d None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:32.779818 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c396be-0034-44fa-876f-5ad456e1784d None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:32.779818 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c396be-0034-44fa-876f-5ad456e1784d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:32.796942 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa75ff87-f12d-47a3-a2ca-ac6fd81eb68f admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:32.801290 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a54550-b7a3-40de-a64c-b41a56209b63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36639dce4f97412f89d71367f650b901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:32.801990 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a54550-b7a3-40de-a64c-b41a56209b63 service cinder] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:32.802278 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a54550-b7a3-40de-a64c-b41a56209b63 service cinder] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:32.802643 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a54550-b7a3-40de-a64c-b41a56209b63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:32.818827 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-a5c396be-0034-44fa-876f-5ad456e1784d None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:41:32.820084 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-a5c396be-0034-44fa-876f-5ad456e1784d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:32.910005 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a54550-b7a3-40de-a64c-b41a56209b63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:32.911669 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a54550-b7a3-40de-a64c-b41a56209b63 service cinder] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:32.913247 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 959/1907] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:41:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:41:32.913906 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c396be-0034-44fa-876f-5ad456e1784d None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:32.916319 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-a5c396be-0034-44fa-876f-5ad456e1784d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:32.923554 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 949/1908] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:41:32 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:41:32.925166 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8b49e6-cdcf-4cdd-abe7-9d8fb030ebcb None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:32.925547 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8b49e6-cdcf-4cdd-abe7-9d8fb030ebcb None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:32.925826 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8b49e6-cdcf-4cdd-abe7-9d8fb030ebcb None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:32.927636 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 950/1909] 10.222.0.47 () {54 vars in 921 bytes} [Wed Aug 24 06:41:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:41:32.928253 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8b49e6-cdcf-4cdd-abe7-9d8fb030ebcb None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:32.932620 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-3e8b49e6-cdcf-4cdd-abe7-9d8fb030ebcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:32.937998 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c396be-0034-44fa-876f-5ad456e1784d None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:32.942060 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e70a55-ebfa-4120-bdbd-c3979f7490c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36639dce4f97412f89d71367f650b901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:32.943059 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e70a55-ebfa-4120-bdbd-c3979f7490c6 service cinder] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:32.943347 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e70a55-ebfa-4120-bdbd-c3979f7490c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:32.944683 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e70a55-ebfa-4120-bdbd-c3979f7490c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:33.014091 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8b49e6-cdcf-4cdd-abe7-9d8fb030ebcb None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:33.017827 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0168d1-ba70-44ad-bbb1-f0eadb49e3ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:33.018455 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0168d1-ba70-44ad-bbb1-f0eadb49e3ee service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:33.023552 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0168d1-ba70-44ad-bbb1-f0eadb49e3ee service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:33.023552 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0168d1-ba70-44ad-bbb1-f0eadb49e3ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:33.025579 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e70a55-ebfa-4120-bdbd-c3979f7490c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:33.027462 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e70a55-ebfa-4120-bdbd-c3979f7490c6 service cinder] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:33.030748 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0168d1-ba70-44ad-bbb1-f0eadb49e3ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:33.032465 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0168d1-ba70-44ad-bbb1-f0eadb49e3ee service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:33.039339 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 960/1910] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:41:32 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:41:33.040913 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b351a62e-ca10-4a28-88af-ffeb375f31c8 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:33.041032 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b351a62e-ca10-4a28-88af-ffeb375f31c8 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:33.041309 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b351a62e-ca10-4a28-88af-ffeb375f31c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:33.045531 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 951/1911] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:41:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:41:33.064455 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-b351a62e-ca10-4a28-88af-ffeb375f31c8 None None] MFA Rules not processed for user `1f4f49c1b08c460f8ba6c451c9c92c2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:41:33.065736 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-b351a62e-ca10-4a28-88af-ffeb375f31c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:33.170933 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 961/1912] 10.222.0.47 () {58 vars in 1018 bytes} [Wed Aug 24 06:41:33 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:41:33.182556 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0168d1-ba70-44ad-bbb1-f0eadb49e3ee service neutron] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:33.185944 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-0f0168d1-ba70-44ad-bbb1-f0eadb49e3ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:33.224346 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0168d1-ba70-44ad-bbb1-f0eadb49e3ee service neutron] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:33.230667 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf936ca3-f0eb-4640-9d10-9d0c74f56729 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4f49c1b08c460f8ba6c451c9c92c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:33.231677 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf936ca3-f0eb-4640-9d10-9d0c74f56729 service nova] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:33.231989 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf936ca3-f0eb-4640-9d10-9d0c74f56729 service nova] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:33.232245 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf936ca3-f0eb-4640-9d10-9d0c74f56729 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:33.252155 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf936ca3-f0eb-4640-9d10-9d0c74f56729 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:33.253870 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf936ca3-f0eb-4640-9d10-9d0c74f56729 service nova] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:33.267487 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 952/1913] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:41:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:41:33.366925 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b351a62e-ca10-4a28-88af-ffeb375f31c8 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:33.380665 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b351a62e-ca10-4a28-88af-ffeb375f31c8 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:33.385118 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562a116a-209c-439e-8fda-a4492a2846cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36639dce4f97412f89d71367f650b901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:33.386115 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562a116a-209c-439e-8fda-a4492a2846cb service cinder] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:33.386709 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562a116a-209c-439e-8fda-a4492a2846cb service cinder] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:33.387711 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562a116a-209c-439e-8fda-a4492a2846cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:33.389976 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-562a116a-209c-439e-8fda-a4492a2846cb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:33.463380 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562a116a-209c-439e-8fda-a4492a2846cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:33.464868 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562a116a-209c-439e-8fda-a4492a2846cb service cinder] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:33.479681 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 962/1914] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:41:33 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:41:34.052909 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf936ca3-f0eb-4640-9d10-9d0c74f56729 service nova] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:34.054618 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-bf936ca3-f0eb-4640-9d10-9d0c74f56729 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:34.078062 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf936ca3-f0eb-4640-9d10-9d0c74f56729 service nova] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:34.087521 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eb9eb0-cffc-4554-b217-ea5882f935f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:34.089077 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eb9eb0-cffc-4554-b217-ea5882f935f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:34.089886 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eb9eb0-cffc-4554-b217-ea5882f935f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:34.092199 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eb9eb0-cffc-4554-b217-ea5882f935f3 admin admin] PATH_INFO: `/v3/projects/952767ebbfa548649beff4719381b354` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:34.094677 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6eb9eb0-cffc-4554-b217-ea5882f935f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=952767ebbfa548649beff4719381b354)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:34.098790 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6eb9eb0-cffc-4554-b217-ea5882f935f3 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:34.397688 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-f6eb9eb0-cffc-4554-b217-ea5882f935f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '952767ebbfa548649beff4719381b354', 'request_id': 'req-f6eb9eb0-cffc-4554-b217-ea5882f935f3'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:34.413392 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-f6eb9eb0-cffc-4554-b217-ea5882f935f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '952767ebbfa548649beff4719381b354', 'request_id': 'req-f6eb9eb0-cffc-4554-b217-ea5882f935f3'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:34.604067 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-f6eb9eb0-cffc-4554-b217-ea5882f935f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '952767ebbfa548649beff4719381b354', 'request_id': 'req-f6eb9eb0-cffc-4554-b217-ea5882f935f3'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:34.618098 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-f6eb9eb0-cffc-4554-b217-ea5882f935f3 admin admin] The token cache is being invalidate because project 952767ebbfa548649beff4719381b354 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:41:34.622771 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-f6eb9eb0-cffc-4554-b217-ea5882f935f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:34.623339 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-f6eb9eb0-cffc-4554-b217-ea5882f935f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:34.625039 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 953/1915] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:41:34 2022] DELETE /identity/v3/projects/952767ebbfa548649beff4719381b354 => generated 0 bytes in 573 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:41:34.629130 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562a116a-209c-439e-8fda-a4492a2846cb service cinder] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:34.640750 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562a116a-209c-439e-8fda-a4492a2846cb service cinder] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:34.709997 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be84c28-7b32-4e39-a301-fd0dda42451b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:34.710570 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be84c28-7b32-4e39-a301-fd0dda42451b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:34.711004 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be84c28-7b32-4e39-a301-fd0dda42451b admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:34.711634 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be84c28-7b32-4e39-a301-fd0dda42451b admin admin] PATH_INFO: `/v3/users/ef0d8a77a8ac41f0ad15c235e84a6024` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:34.712768 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be84c28-7b32-4e39-a301-fd0dda42451b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef0d8a77a8ac41f0ad15c235e84a6024)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:34.714806 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be84c28-7b32-4e39-a301-fd0dda42451b admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:34.982891 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-9be84c28-7b32-4e39-a301-fd0dda42451b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef0d8a77a8ac41f0ad15c235e84a6024'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:34.983602 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-9be84c28-7b32-4e39-a301-fd0dda42451b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef0d8a77a8ac41f0ad15c235e84a6024'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:34.984215 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-9be84c28-7b32-4e39-a301-fd0dda42451b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef0d8a77a8ac41f0ad15c235e84a6024'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:35.083677 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-9be84c28-7b32-4e39-a301-fd0dda42451b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef0d8a77a8ac41f0ad15c235e84a6024'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:35.097265 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-9be84c28-7b32-4e39-a301-fd0dda42451b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef0d8a77a8ac41f0ad15c235e84a6024'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:35.112204 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 963/1916] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:41:34 2022] DELETE /identity/v3/users/ef0d8a77a8ac41f0ad15c235e84a6024 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:41:35.435007 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eb9eb0-cffc-4554-b217-ea5882f935f3 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:35.437319 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-f6eb9eb0-cffc-4554-b217-ea5882f935f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:35.529644 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eb9eb0-cffc-4554-b217-ea5882f935f3 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:35.614785 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fec4e98-ce3a-4c90-b912-04a5c014b2f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36639dce4f97412f89d71367f650b901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:35.615608 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fec4e98-ce3a-4c90-b912-04a5c014b2f3 service cinder] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:35.615902 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fec4e98-ce3a-4c90-b912-04a5c014b2f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:35.616245 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fec4e98-ce3a-4c90-b912-04a5c014b2f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:35.619610 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-1fec4e98-ce3a-4c90-b912-04a5c014b2f3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:35.634669 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:41:35.645751 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be84c28-7b32-4e39-a301-fd0dda42451b admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:35.649903 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-9be84c28-7b32-4e39-a301-fd0dda42451b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:35.764629 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be84c28-7b32-4e39-a301-fd0dda42451b admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:35.771052 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fec4e98-ce3a-4c90-b912-04a5c014b2f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:35.774349 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fec4e98-ce3a-4c90-b912-04a5c014b2f3 service cinder] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:35.778207 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dda552e-507c-4f7c-83a2-6d41e31da1b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:35.782357 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dda552e-507c-4f7c-83a2-6d41e31da1b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:35.782651 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dda552e-507c-4f7c-83a2-6d41e31da1b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:35.782935 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dda552e-507c-4f7c-83a2-6d41e31da1b1 admin admin] PATH_INFO: `/v3/projects/8514dfa63c584b99892f05f2099572b9` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:35.785719 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dda552e-507c-4f7c-83a2-6d41e31da1b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8514dfa63c584b99892f05f2099572b9)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:35.787961 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 954/1917] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:41:35 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:41:35.788560 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dda552e-507c-4f7c-83a2-6d41e31da1b1 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:36.016986 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-1dda552e-507c-4f7c-83a2-6d41e31da1b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8514dfa63c584b99892f05f2099572b9', 'request_id': 'req-1dda552e-507c-4f7c-83a2-6d41e31da1b1'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:36.158228 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:41:36.158912 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-1dda552e-507c-4f7c-83a2-6d41e31da1b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8514dfa63c584b99892f05f2099572b9', 'request_id': 'req-1dda552e-507c-4f7c-83a2-6d41e31da1b1'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:36.166330 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-1dda552e-507c-4f7c-83a2-6d41e31da1b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8514dfa63c584b99892f05f2099572b9', 'request_id': 'req-1dda552e-507c-4f7c-83a2-6d41e31da1b1'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:36.178836 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-1dda552e-507c-4f7c-83a2-6d41e31da1b1 admin admin] The token cache is being invalidate because project 8514dfa63c584b99892f05f2099572b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81723) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:41:36.179170 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-1dda552e-507c-4f7c-83a2-6d41e31da1b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:36.179778 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-1dda552e-507c-4f7c-83a2-6d41e31da1b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:36.181332 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 964/1918] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:41:35 2022] DELETE /identity/v3/projects/8514dfa63c584b99892f05f2099572b9 => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:41:36.187223 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ba735c-9bbf-4a03-bae3-a6ed451a8cff None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:36.187538 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ba735c-9bbf-4a03-bae3-a6ed451a8cff None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:36.187842 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ba735c-9bbf-4a03-bae3-a6ed451a8cff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:36.207787 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-03ba735c-9bbf-4a03-bae3-a6ed451a8cff None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:41:36.208996 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-03ba735c-9bbf-4a03-bae3-a6ed451a8cff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:36.273166 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 955/1919] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:41:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:41:36.281768 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796ff5d2-4d98-473c-9d64-64b9c25a2a04 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:36.282137 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796ff5d2-4d98-473c-9d64-64b9c25a2a04 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:36.282498 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796ff5d2-4d98-473c-9d64-64b9c25a2a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:36.326322 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-796ff5d2-4d98-473c-9d64-64b9c25a2a04 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:41:36.327715 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-796ff5d2-4d98-473c-9d64-64b9c25a2a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:36.343872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 965/1920] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:41:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:41:36.348968 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ba735c-9bbf-4a03-bae3-a6ed451a8cff None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:36.363080 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ba735c-9bbf-4a03-bae3-a6ed451a8cff None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:36.367934 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf5b2ea-fc39-4fad-a95f-90955e0f83dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:36.376458 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf5b2ea-fc39-4fad-a95f-90955e0f83dd admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:36.376458 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf5b2ea-fc39-4fad-a95f-90955e0f83dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:36.376458 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf5b2ea-fc39-4fad-a95f-90955e0f83dd admin admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:36.376458 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf5b2ea-fc39-4fad-a95f-90955e0f83dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:41:36.376458 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf5b2ea-fc39-4fad-a95f-90955e0f83dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:36.376458 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf5b2ea-fc39-4fad-a95f-90955e0f83dd admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:36.384914 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 956/1921] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 24 06:41:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:41:36.390082 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796ff5d2-4d98-473c-9d64-64b9c25a2a04 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:36.399999 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796ff5d2-4d98-473c-9d64-64b9c25a2a04 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:36.409094 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b46461-72f2-4d9b-8878-fe9982be4a9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:36.409777 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b46461-72f2-4d9b-8878-fe9982be4a9b admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:36.410055 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b46461-72f2-4d9b-8878-fe9982be4a9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:36.410325 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b46461-72f2-4d9b-8878-fe9982be4a9b admin admin] PATH_INFO: `/v3/projects` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:36.410953 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b46461-72f2-4d9b-8878-fe9982be4a9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:36.412929 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b46461-72f2-4d9b-8878-fe9982be4a9b admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:36.564077 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 966/1922] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:41:36 2022] POST /identity/v3/projects => generated 379 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:41:36.565971 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf5b2ea-fc39-4fad-a95f-90955e0f83dd admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:36.579992 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf5b2ea-fc39-4fad-a95f-90955e0f83dd admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:36.676413 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee82e344-fa8d-42cf-8284-fd4fb4814b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:36.677032 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee82e344-fa8d-42cf-8284-fd4fb4814b78 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:36.677353 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee82e344-fa8d-42cf-8284-fd4fb4814b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:36.677626 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee82e344-fa8d-42cf-8284-fd4fb4814b78 admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:36.678246 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee82e344-fa8d-42cf-8284-fd4fb4814b78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:36.680173 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee82e344-fa8d-42cf-8284-fd4fb4814b78 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:36.766828 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 957/1923] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:41:36 2022] POST /identity/v3/users => generated 290 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:41:36.775286 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b46461-72f2-4d9b-8878-fe9982be4a9b admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:36.786485 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee82e344-fa8d-42cf-8284-fd4fb4814b78 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:36.789412 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-ee82e344-fa8d-42cf-8284-fd4fb4814b78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:36.791163 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b46461-72f2-4d9b-8878-fe9982be4a9b admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:36.798280 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5437fa40-1f0a-4699-97c6-6d043ed38a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:36.798856 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5437fa40-1f0a-4699-97c6-6d043ed38a96 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:36.799179 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5437fa40-1f0a-4699-97c6-6d043ed38a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:36.799348 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5437fa40-1f0a-4699-97c6-6d043ed38a96 admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:36.800117 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5437fa40-1f0a-4699-97c6-6d043ed38a96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:36.802249 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5437fa40-1f0a-4699-97c6-6d043ed38a96 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:36.807893 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee82e344-fa8d-42cf-8284-fd4fb4814b78 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:36.816543 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 967/1924] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:41:36 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:41:36.816938 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591f3509-d965-4e13-88a5-c048ca1d55d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:36.817907 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f3509-d965-4e13-88a5-c048ca1d55d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:36.818198 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f3509-d965-4e13-88a5-c048ca1d55d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:36.818471 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f3509-d965-4e13-88a5-c048ca1d55d5 admin admin] PATH_INFO: `/v3/users/d4ae2c4920d84bc98b176f88a6dd0f07` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:36.821509 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5437fa40-1f0a-4699-97c6-6d043ed38a96 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:36.821665 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591f3509-d965-4e13-88a5-c048ca1d55d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4ae2c4920d84bc98b176f88a6dd0f07)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:36.823570 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591f3509-d965-4e13-88a5-c048ca1d55d5 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:36.833011 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5437fa40-1f0a-4699-97c6-6d043ed38a96 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:36.838590 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0f36d3-a644-4aa5-96a7-c5ae5edeb12b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:36.839221 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0f36d3-a644-4aa5-96a7-c5ae5edeb12b admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:36.840787 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0f36d3-a644-4aa5-96a7-c5ae5edeb12b admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:36.840787 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0f36d3-a644-4aa5-96a7-c5ae5edeb12b admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:36.842657 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0f36d3-a644-4aa5-96a7-c5ae5edeb12b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:36.843466 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0f36d3-a644-4aa5-96a7-c5ae5edeb12b admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:36.855369 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 968/1925] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:41:36 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:41:36.860417 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0f36d3-a644-4aa5-96a7-c5ae5edeb12b admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:36.875608 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0f36d3-a644-4aa5-96a7-c5ae5edeb12b admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:36.889249 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb05889-e519-4c5f-aef8-4c612afc518c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:36.890337 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb05889-e519-4c5f-aef8-4c612afc518c admin admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:36.891316 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb05889-e519-4c5f-aef8-4c612afc518c admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:36.891601 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb05889-e519-4c5f-aef8-4c612afc518c admin admin] PATH_INFO: `/v3/projects/e676a17baa754a9fa4eb6ee91149c0b2/users/cb65b909126b42af88786fa4e6c793a1/roles/e1f9b167c51c4a388cbc9594db8a465d` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:36.912368 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb05889-e519-4c5f-aef8-4c612afc518c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e676a17baa754a9fa4eb6ee91149c0b2, user_id=cb65b909126b42af88786fa4e6c793a1, role_id=e1f9b167c51c4a388cbc9594db8a465d)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:36.918581 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb05889-e519-4c5f-aef8-4c612afc518c admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:37.079038 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 969/1926] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:41:36 2022] PUT /identity/v3/projects/e676a17baa754a9fa4eb6ee91149c0b2/users/cb65b909126b42af88786fa4e6c793a1/roles/e1f9b167c51c4a388cbc9594db8a465d => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:41:37.082229 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900ed383-3c13-4f3f-b98d-8926d94e16f4 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:37.082672 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900ed383-3c13-4f3f-b98d-8926d94e16f4 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:37.082875 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900ed383-3c13-4f3f-b98d-8926d94e16f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:37.124440 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-900ed383-3c13-4f3f-b98d-8926d94e16f4 None None] MFA Rules not processed for user `cb65b909126b42af88786fa4e6c793a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:41:37.125801 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-900ed383-3c13-4f3f-b98d-8926d94e16f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:37.158936 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-591f3509-d965-4e13-88a5-c048ca1d55d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4ae2c4920d84bc98b176f88a6dd0f07'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:37.159804 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-591f3509-d965-4e13-88a5-c048ca1d55d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4ae2c4920d84bc98b176f88a6dd0f07'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:37.161531 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-591f3509-d965-4e13-88a5-c048ca1d55d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4ae2c4920d84bc98b176f88a6dd0f07'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:37.178258 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-591f3509-d965-4e13-88a5-c048ca1d55d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4ae2c4920d84bc98b176f88a6dd0f07'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:37.212092 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 970/1927] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:41:37 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:41:37.218630 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900ed383-3c13-4f3f-b98d-8926d94e16f4 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:37.220592 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-900ed383-3c13-4f3f-b98d-8926d94e16f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:37.281472 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-591f3509-d965-4e13-88a5-c048ca1d55d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4ae2c4920d84bc98b176f88a6dd0f07'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:37.297530 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900ed383-3c13-4f3f-b98d-8926d94e16f4 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:37.301140 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 958/1928] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:41:36 2022] DELETE /identity/v3/users/d4ae2c4920d84bc98b176f88a6dd0f07 => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:41:37.302816 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94a1083-4f0c-4b82-b768-3b6dabbbfe32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:37.303487 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94a1083-4f0c-4b82-b768-3b6dabbbfe32 admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:37.303897 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94a1083-4f0c-4b82-b768-3b6dabbbfe32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:37.304162 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94a1083-4f0c-4b82-b768-3b6dabbbfe32 admin admin] PATH_INFO: `/v3/projects` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:37.304789 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94a1083-4f0c-4b82-b768-3b6dabbbfe32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:37.306161 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a286c5-d570-412a-bdad-16dfa004be2c None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:37.306450 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a286c5-d570-412a-bdad-16dfa004be2c None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:37.306729 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a286c5-d570-412a-bdad-16dfa004be2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:37.306824 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94a1083-4f0c-4b82-b768-3b6dabbbfe32 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:37.329575 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-b8a286c5-d570-412a-bdad-16dfa004be2c None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:41:37.331108 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-b8a286c5-d570-412a-bdad-16dfa004be2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:37.395953 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 959/1929] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:41:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:41:37.405948 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a286c5-d570-412a-bdad-16dfa004be2c None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:37.407357 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-b8a286c5-d570-412a-bdad-16dfa004be2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:37.463148 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a286c5-d570-412a-bdad-16dfa004be2c None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:37.466811 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8851c94-5aa7-40f5-852e-61ff951012f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:37.467510 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8851c94-5aa7-40f5-852e-61ff951012f3 service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:37.467786 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8851c94-5aa7-40f5-852e-61ff951012f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:37.468056 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8851c94-5aa7-40f5-852e-61ff951012f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:37.477463 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 971/1930] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:41:37 2022] POST /identity/v3/projects => generated 377 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:41:37.478902 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94a1083-4f0c-4b82-b768-3b6dabbbfe32 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:37.482291 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8851c94-5aa7-40f5-852e-61ff951012f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:37.493591 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8851c94-5aa7-40f5-852e-61ff951012f3 service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:37.493937 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 960/1931] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:41:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:41:37.556077 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94a1083-4f0c-4b82-b768-3b6dabbbfe32 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:37.561746 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e288784-fdad-4cd7-a28d-9d4c8d8fd442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:37.562420 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e288784-fdad-4cd7-a28d-9d4c8d8fd442 admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:37.562758 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e288784-fdad-4cd7-a28d-9d4c8d8fd442 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:37.563035 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e288784-fdad-4cd7-a28d-9d4c8d8fd442 admin admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:37.563668 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e288784-fdad-4cd7-a28d-9d4c8d8fd442 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:37.565722 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e288784-fdad-4cd7-a28d-9d4c8d8fd442 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:37.715046 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 972/1932] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:41:37 2022] POST /identity/v3/users => generated 289 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:41:37.718814 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8851c94-5aa7-40f5-852e-61ff951012f3 service neutron] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:37.732314 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8851c94-5aa7-40f5-852e-61ff951012f3 service neutron] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:37.737007 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cb3612-0329-4dd3-931e-dbcdce710d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:37.737883 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cb3612-0329-4dd3-931e-dbcdce710d2d admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:37.738165 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cb3612-0329-4dd3-931e-dbcdce710d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:37.738438 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cb3612-0329-4dd3-931e-dbcdce710d2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:37.739084 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cb3612-0329-4dd3-931e-dbcdce710d2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:37.745048 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cb3612-0329-4dd3-931e-dbcdce710d2d admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:37.752061 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 961/1933] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:41:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:41:37.756311 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e288784-fdad-4cd7-a28d-9d4c8d8fd442 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:37.767541 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e288784-fdad-4cd7-a28d-9d4c8d8fd442 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:37.772484 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8042e4f9-4b9d-44ca-b3f8-d68af1bfb733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:37.773102 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8042e4f9-4b9d-44ca-b3f8-d68af1bfb733 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:37.773535 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8042e4f9-4b9d-44ca-b3f8-d68af1bfb733 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:37.773855 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8042e4f9-4b9d-44ca-b3f8-d68af1bfb733 admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:37.774515 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8042e4f9-4b9d-44ca-b3f8-d68af1bfb733 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:37.776106 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8042e4f9-4b9d-44ca-b3f8-d68af1bfb733 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:37.790466 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 973/1934] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:41:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:41:37.791798 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cb3612-0329-4dd3-931e-dbcdce710d2d admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:37.803165 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cb3612-0329-4dd3-931e-dbcdce710d2d admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:37.808169 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49702b1f-c04f-461b-84bb-3ebe5c006fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:37.810017 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49702b1f-c04f-461b-84bb-3ebe5c006fcf admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:37.810582 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49702b1f-c04f-461b-84bb-3ebe5c006fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:37.811121 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49702b1f-c04f-461b-84bb-3ebe5c006fcf admin admin] PATH_INFO: `/v3/projects/ad1fcd639e604251a43bd90a5a7ab2a5/users/943205be4a664d37bc8608ce4bd9b4cf/roles/e1f9b167c51c4a388cbc9594db8a465d` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:37.831872 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49702b1f-c04f-461b-84bb-3ebe5c006fcf admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad1fcd639e604251a43bd90a5a7ab2a5, user_id=943205be4a664d37bc8608ce4bd9b4cf, role_id=e1f9b167c51c4a388cbc9594db8a465d)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:37.836433 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49702b1f-c04f-461b-84bb-3ebe5c006fcf admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:38.074527 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 962/1935] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:41:37 2022] PUT /identity/v3/projects/ad1fcd639e604251a43bd90a5a7ab2a5/users/943205be4a664d37bc8608ce4bd9b4cf/roles/e1f9b167c51c4a388cbc9594db8a465d => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:41:38.093135 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7fefef-3706-414e-92c4-82be7e910469 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:38.095369 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7fefef-3706-414e-92c4-82be7e910469 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:38.095369 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7fefef-3706-414e-92c4-82be7e910469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:38.144570 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49702b1f-c04f-461b-84bb-3ebe5c006fcf admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:38.147594 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-49702b1f-c04f-461b-84bb-3ebe5c006fcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:38.155066 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-7b7fefef-3706-414e-92c4-82be7e910469 None None] MFA Rules not processed for user `943205be4a664d37bc8608ce4bd9b4cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:41:38.158249 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-7b7fefef-3706-414e-92c4-82be7e910469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:38.274271 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49702b1f-c04f-461b-84bb-3ebe5c006fcf admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:38.286841 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfdd72c-0580-4218-a13e-477915e03112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:38.287669 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfdd72c-0580-4218-a13e-477915e03112 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:38.288058 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfdd72c-0580-4218-a13e-477915e03112 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:38.288345 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfdd72c-0580-4218-a13e-477915e03112 admin admin] PATH_INFO: `/v3/projects/fef584f471254e6790f9f26f31b29543` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:38.289692 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfdd72c-0580-4218-a13e-477915e03112 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fef584f471254e6790f9f26f31b29543)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:38.294362 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfdd72c-0580-4218-a13e-477915e03112 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:38.303772 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 974/1936] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:41:38 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:41:38.305715 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7fefef-3706-414e-92c4-82be7e910469 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:38.328891 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7fefef-3706-414e-92c4-82be7e910469 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:38.337024 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e14c8a5-be30-4db2-b9f6-61254e221ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:38.337965 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e14c8a5-be30-4db2-b9f6-61254e221ca5 admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:38.338562 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e14c8a5-be30-4db2-b9f6-61254e221ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:38.339366 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e14c8a5-be30-4db2-b9f6-61254e221ca5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:38.340966 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e14c8a5-be30-4db2-b9f6-61254e221ca5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:38.344053 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e14c8a5-be30-4db2-b9f6-61254e221ca5 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:38.528428 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 975/1937] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:41:38 2022] POST /identity/v3/projects => generated 377 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:41:38.530319 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e14c8a5-be30-4db2-b9f6-61254e221ca5 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:38.533825 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-7e14c8a5-be30-4db2-b9f6-61254e221ca5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:38.538536 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-3bfdd72c-0580-4218-a13e-477915e03112 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fef584f471254e6790f9f26f31b29543', 'request_id': 'req-3bfdd72c-0580-4218-a13e-477915e03112'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:38.546284 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-3bfdd72c-0580-4218-a13e-477915e03112 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fef584f471254e6790f9f26f31b29543', 'request_id': 'req-3bfdd72c-0580-4218-a13e-477915e03112'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:38.611235 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e14c8a5-be30-4db2-b9f6-61254e221ca5 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:38.616013 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22041a19-a40f-4d44-859a-9dd568a200b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:38.616700 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22041a19-a40f-4d44-859a-9dd568a200b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:38.617050 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22041a19-a40f-4d44-859a-9dd568a200b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:38.617428 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22041a19-a40f-4d44-859a-9dd568a200b7 admin admin] PATH_INFO: `/v3/users/e5b9c1167f1f4a77b94c0e6843c52ef4` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:38.618577 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22041a19-a40f-4d44-859a-9dd568a200b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5b9c1167f1f4a77b94c0e6843c52ef4)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:38.620511 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22041a19-a40f-4d44-859a-9dd568a200b7 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:38.780249 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-3bfdd72c-0580-4218-a13e-477915e03112 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fef584f471254e6790f9f26f31b29543', 'request_id': 'req-3bfdd72c-0580-4218-a13e-477915e03112'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:38.804006 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-3bfdd72c-0580-4218-a13e-477915e03112 admin admin] The token cache is being invalidate because project fef584f471254e6790f9f26f31b29543 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:41:38.804475 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-3bfdd72c-0580-4218-a13e-477915e03112 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:38.805421 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-3bfdd72c-0580-4218-a13e-477915e03112 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:38.808122 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 963/1938] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:41:38 2022] DELETE /identity/v3/projects/fef584f471254e6790f9f26f31b29543 => generated 0 bytes in 664 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:41:38.810354 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfdd72c-0580-4218-a13e-477915e03112 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:38.812639 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-3bfdd72c-0580-4218-a13e-477915e03112 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:38.892918 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfdd72c-0580-4218-a13e-477915e03112 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:38.898097 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958c3b4e-ee48-466c-88ce-ed9f9d8ebeb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:38.898726 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958c3b4e-ee48-466c-88ce-ed9f9d8ebeb7 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:38.899016 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958c3b4e-ee48-466c-88ce-ed9f9d8ebeb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:38.899300 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958c3b4e-ee48-466c-88ce-ed9f9d8ebeb7 admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:38.899964 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958c3b4e-ee48-466c-88ce-ed9f9d8ebeb7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:38.901974 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958c3b4e-ee48-466c-88ce-ed9f9d8ebeb7 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:38.958974 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-22041a19-a40f-4d44-859a-9dd568a200b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5b9c1167f1f4a77b94c0e6843c52ef4'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:38.959548 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-22041a19-a40f-4d44-859a-9dd568a200b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5b9c1167f1f4a77b94c0e6843c52ef4'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:38.960137 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-22041a19-a40f-4d44-859a-9dd568a200b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5b9c1167f1f4a77b94c0e6843c52ef4'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:39.068260 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 964/1939] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:41:38 2022] POST /identity/v3/users => generated 295 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:41:39.068260 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa818ce-69f3-45b9-8add-f7192c2b5b67 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:39.068260 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa818ce-69f3-45b9-8add-f7192c2b5b67 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:39.068260 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa818ce-69f3-45b9-8add-f7192c2b5b67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:39.087414 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-faa818ce-69f3-45b9-8add-f7192c2b5b67 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:41:39.088632 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-faa818ce-69f3-45b9-8add-f7192c2b5b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:39.096828 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 965/1940] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:41:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:41:39.097650 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa818ce-69f3-45b9-8add-f7192c2b5b67 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:39.098875 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-faa818ce-69f3-45b9-8add-f7192c2b5b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:39.111704 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa818ce-69f3-45b9-8add-f7192c2b5b67 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:39.116541 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-22041a19-a40f-4d44-859a-9dd568a200b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5b9c1167f1f4a77b94c0e6843c52ef4'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:39.116683 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83cac5d-9bcf-4cb4-be97-fca1d8afd631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:39.117161 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83cac5d-9bcf-4cb4-be97-fca1d8afd631 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:39.117470 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83cac5d-9bcf-4cb4-be97-fca1d8afd631 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:39.117744 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83cac5d-9bcf-4cb4-be97-fca1d8afd631 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:39.118395 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83cac5d-9bcf-4cb4-be97-fca1d8afd631 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:39.120025 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83cac5d-9bcf-4cb4-be97-fca1d8afd631 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:39.132013 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 966/1941] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:41:39 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:41:39.133231 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bbe0b6-b1a9-49fb-9f2d-0c2bfa44b1f8 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:39.133515 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bbe0b6-b1a9-49fb-9f2d-0c2bfa44b1f8 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:39.133790 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bbe0b6-b1a9-49fb-9f2d-0c2bfa44b1f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:39.151002 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-22041a19-a40f-4d44-859a-9dd568a200b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5b9c1167f1f4a77b94c0e6843c52ef4'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:39.158607 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-11bbe0b6-b1a9-49fb-9f2d-0c2bfa44b1f8 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:41:39.159723 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-11bbe0b6-b1a9-49fb-9f2d-0c2bfa44b1f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:39.166196 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 976/1942] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:41:38 2022] DELETE /identity/v3/users/e5b9c1167f1f4a77b94c0e6843c52ef4 => generated 0 bytes in 637 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:41:39.167116 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22041a19-a40f-4d44-859a-9dd568a200b7 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:39.173567 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 967/1943] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:41:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:41:39.174956 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9447f222-b63c-4e8e-8cfe-df4e9472bf23 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:39.175318 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9447f222-b63c-4e8e-8cfe-df4e9472bf23 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:39.175657 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9447f222-b63c-4e8e-8cfe-df4e9472bf23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:39.198077 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-9447f222-b63c-4e8e-8cfe-df4e9472bf23 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:41:39.200153 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-9447f222-b63c-4e8e-8cfe-df4e9472bf23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:39.281688 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 968/1944] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:41:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:41:39.282720 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9447f222-b63c-4e8e-8cfe-df4e9472bf23 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:39.287072 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22041a19-a40f-4d44-859a-9dd568a200b7 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:39.294054 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88afa0b-5c6e-47da-8782-3209e55e743f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:39.294990 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88afa0b-5c6e-47da-8782-3209e55e743f admin admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:39.295416 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88afa0b-5c6e-47da-8782-3209e55e743f admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:39.295837 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88afa0b-5c6e-47da-8782-3209e55e743f admin admin] PATH_INFO: `/v3/projects/92c23c7df11e4f259b7a4d9b9facccab/users/091e7cea39ff42c9b730907d12b210cc/roles/486144ad1bcd417e86acdbc1d14327bf` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:39.301320 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9447f222-b63c-4e8e-8cfe-df4e9472bf23 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:39.307293 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3588085-eec1-42f9-bf4b-a37ad4cd70bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:39.307632 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3588085-eec1-42f9-bf4b-a37ad4cd70bf admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:39.312499 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3588085-eec1-42f9-bf4b-a37ad4cd70bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:39.312499 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3588085-eec1-42f9-bf4b-a37ad4cd70bf admin admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:39.312499 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3588085-eec1-42f9-bf4b-a37ad4cd70bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:41:39.312499 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3588085-eec1-42f9-bf4b-a37ad4cd70bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:39.312499 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3588085-eec1-42f9-bf4b-a37ad4cd70bf admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:39.316054 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88afa0b-5c6e-47da-8782-3209e55e743f admin admin] RBAC: Authorizing `identity:create_grant(project_id=92c23c7df11e4f259b7a4d9b9facccab, user_id=091e7cea39ff42c9b730907d12b210cc, role_id=486144ad1bcd417e86acdbc1d14327bf)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:39.319022 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88afa0b-5c6e-47da-8782-3209e55e743f admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:39.332585 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 969/1945] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 24 06:41:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:41:39.333819 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3588085-eec1-42f9-bf4b-a37ad4cd70bf admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:39.336110 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-b3588085-eec1-42f9-bf4b-a37ad4cd70bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:39.404670 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3588085-eec1-42f9-bf4b-a37ad4cd70bf admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:39.408786 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cb3079-d3c5-4693-8a80-7556aeb7295f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:39.409569 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cb3079-d3c5-4693-8a80-7556aeb7295f service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:39.409954 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cb3079-d3c5-4693-8a80-7556aeb7295f service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:39.410246 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cb3079-d3c5-4693-8a80-7556aeb7295f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:39.420648 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cb3079-d3c5-4693-8a80-7556aeb7295f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:39.422323 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cb3079-d3c5-4693-8a80-7556aeb7295f service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:39.435499 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 970/1946] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:41:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:41:39.436836 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cb3079-d3c5-4693-8a80-7556aeb7295f service neutron] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:39.449949 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cb3079-d3c5-4693-8a80-7556aeb7295f service neutron] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:39.455293 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dace7a-6a04-4d6d-8d99-56aba438b1da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:39.456270 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dace7a-6a04-4d6d-8d99-56aba438b1da admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:39.456566 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dace7a-6a04-4d6d-8d99-56aba438b1da admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:39.456843 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dace7a-6a04-4d6d-8d99-56aba438b1da admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:39.457670 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dace7a-6a04-4d6d-8d99-56aba438b1da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:39.459810 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dace7a-6a04-4d6d-8d99-56aba438b1da admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:39.501291 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 977/1947] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:41:39 2022] PUT /identity/v3/projects/92c23c7df11e4f259b7a4d9b9facccab/users/091e7cea39ff42c9b730907d12b210cc/roles/486144ad1bcd417e86acdbc1d14327bf => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:41:39.503900 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88afa0b-5c6e-47da-8782-3209e55e743f admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:39.613724 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88afa0b-5c6e-47da-8782-3209e55e743f admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:39.619800 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc72f70-5b45-4c32-891e-21d57a628c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:39.621638 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc72f70-5b45-4c32-891e-21d57a628c37 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:39.624804 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc72f70-5b45-4c32-891e-21d57a628c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:39.624804 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc72f70-5b45-4c32-891e-21d57a628c37 admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:39.624804 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc72f70-5b45-4c32-891e-21d57a628c37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:39.625719 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc72f70-5b45-4c32-891e-21d57a628c37 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:39.644883 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 978/1948] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:41:39 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:41:39.646507 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc72f70-5b45-4c32-891e-21d57a628c37 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:39.656027 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 971/1949] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:41:39 2022] POST /identity/v3/projects => generated 375 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:41:39.661546 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dace7a-6a04-4d6d-8d99-56aba438b1da admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:39.685616 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc72f70-5b45-4c32-891e-21d57a628c37 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:39.789003 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6517c0-f76b-4bbb-b4f3-f83634fa9178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:39.790154 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6517c0-f76b-4bbb-b4f3-f83634fa9178 admin admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:39.790600 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6517c0-f76b-4bbb-b4f3-f83634fa9178 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:39.791048 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6517c0-f76b-4bbb-b4f3-f83634fa9178 admin admin] PATH_INFO: `/v3/projects/92c23c7df11e4f259b7a4d9b9facccab/users/091e7cea39ff42c9b730907d12b210cc/roles/486144ad1bcd417e86acdbc1d14327bf` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:39.794678 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6517c0-f76b-4bbb-b4f3-f83634fa9178 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92c23c7df11e4f259b7a4d9b9facccab, user_id=091e7cea39ff42c9b730907d12b210cc, role_id=486144ad1bcd417e86acdbc1d14327bf)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:39.794887 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dace7a-6a04-4d6d-8d99-56aba438b1da admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:39.797704 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6517c0-f76b-4bbb-b4f3-f83634fa9178 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:39.803217 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e640738-2f3c-4346-828b-aadb7f507e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:39.804924 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e640738-2f3c-4346-828b-aadb7f507e85 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:39.805244 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e640738-2f3c-4346-828b-aadb7f507e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:39.806859 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e640738-2f3c-4346-828b-aadb7f507e85 admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:39.807650 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e640738-2f3c-4346-828b-aadb7f507e85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:39.811185 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e640738-2f3c-4346-828b-aadb7f507e85 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:39.811818 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 979/1950] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:41:39 2022] PUT /identity/v3/projects/92c23c7df11e4f259b7a4d9b9facccab/users/091e7cea39ff42c9b730907d12b210cc/roles/486144ad1bcd417e86acdbc1d14327bf => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:41:39.829722 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a324cfe-d572-483b-9d63-10028de92967 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:39.830229 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a324cfe-d572-483b-9d63-10028de92967 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:39.830722 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a324cfe-d572-483b-9d63-10028de92967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:39.842425 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:41:39.866537 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-2a324cfe-d572-483b-9d63-10028de92967 None None] MFA Rules not processed for user `091e7cea39ff42c9b730907d12b210cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:41:39.867788 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-2a324cfe-d572-483b-9d63-10028de92967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:39.930477 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 972/1951] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:41:39 2022] POST /identity/v3/users => generated 288 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:41:39.935368 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e640738-2f3c-4346-828b-aadb7f507e85 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:39.957439 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e640738-2f3c-4346-828b-aadb7f507e85 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:39.972151 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 980/1952] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:41:39 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:41:40.024835 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fb05bf-abd2-4f3f-87aa-0609f5b8005d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:40.025598 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fb05bf-abd2-4f3f-87aa-0609f5b8005d admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:40.025848 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fb05bf-abd2-4f3f-87aa-0609f5b8005d admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:40.026125 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fb05bf-abd2-4f3f-87aa-0609f5b8005d admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:40.026814 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fb05bf-abd2-4f3f-87aa-0609f5b8005d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:40.028652 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fb05bf-abd2-4f3f-87aa-0609f5b8005d admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:40.039733 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 973/1953] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:41:39 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:41:40.044892 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a324cfe-d572-483b-9d63-10028de92967 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:40.056542 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a324cfe-d572-483b-9d63-10028de92967 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:40.062011 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cf9eaf-b6bf-46f8-8f85-e64281ffcb85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:40.062634 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cf9eaf-b6bf-46f8-8f85-e64281ffcb85 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:40.062912 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cf9eaf-b6bf-46f8-8f85-e64281ffcb85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:40.063183 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cf9eaf-b6bf-46f8-8f85-e64281ffcb85 admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:40.063826 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cf9eaf-b6bf-46f8-8f85-e64281ffcb85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:40.065728 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cf9eaf-b6bf-46f8-8f85-e64281ffcb85 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:40.076775 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 981/1954] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:41:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:41:40.080937 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fb05bf-abd2-4f3f-87aa-0609f5b8005d admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:40.092514 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fb05bf-abd2-4f3f-87aa-0609f5b8005d admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:40.112000 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cf9eaf-b6bf-46f8-8f85-e64281ffcb85 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:40.114919 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-d8cf9eaf-b6bf-46f8-8f85-e64281ffcb85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:40.119964 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0adfa21-cb59-43bd-a7ea-087b7a7d620a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:40.120894 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0adfa21-cb59-43bd-a7ea-087b7a7d620a admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:40.121727 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0adfa21-cb59-43bd-a7ea-087b7a7d620a admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:40.122285 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0adfa21-cb59-43bd-a7ea-087b7a7d620a admin admin] PATH_INFO: `/v3/projects/dc3b812313284d40884e1bd0d21f7575/users/dba7c5b5bb0848caaa5e139fd711c617/roles/e1f9b167c51c4a388cbc9594db8a465d` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:40.141041 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cf9eaf-b6bf-46f8-8f85-e64281ffcb85 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:40.144871 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0adfa21-cb59-43bd-a7ea-087b7a7d620a admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc3b812313284d40884e1bd0d21f7575, user_id=dba7c5b5bb0848caaa5e139fd711c617, role_id=e1f9b167c51c4a388cbc9594db8a465d)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:40.147492 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9ff7d3-477b-4216-a829-a8cdfa9e508d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:40.148391 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9ff7d3-477b-4216-a829-a8cdfa9e508d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:40.148663 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9ff7d3-477b-4216-a829-a8cdfa9e508d admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:40.148938 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9ff7d3-477b-4216-a829-a8cdfa9e508d admin admin] PATH_INFO: `/v3/projects/87571c73f1b442b58d174fc3b6adc8b5` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:40.149827 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0adfa21-cb59-43bd-a7ea-087b7a7d620a admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:40.150130 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9ff7d3-477b-4216-a829-a8cdfa9e508d admin admin] RBAC: Authorizing `identity:delete_project(project_id=87571c73f1b442b58d174fc3b6adc8b5)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:40.152545 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9ff7d3-477b-4216-a829-a8cdfa9e508d admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:40.182435 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:41:40.262120 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 974/1955] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:41:40 2022] PUT /identity/v3/projects/dc3b812313284d40884e1bd0d21f7575/users/dba7c5b5bb0848caaa5e139fd711c617/roles/e1f9b167c51c4a388cbc9594db8a465d => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:41:40.276398 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52924ff5-b1f7-4520-80d3-957a3cd03b0c None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:40.276693 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52924ff5-b1f7-4520-80d3-957a3cd03b0c None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:40.276965 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52924ff5-b1f7-4520-80d3-957a3cd03b0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:40.308744 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-52924ff5-b1f7-4520-80d3-957a3cd03b0c None None] MFA Rules not processed for user `dba7c5b5bb0848caaa5e139fd711c617`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:41:40.310314 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-52924ff5-b1f7-4520-80d3-957a3cd03b0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:40.360509 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-ae9ff7d3-477b-4216-a829-a8cdfa9e508d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87571c73f1b442b58d174fc3b6adc8b5', 'request_id': 'req-ae9ff7d3-477b-4216-a829-a8cdfa9e508d'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:40.403627 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 975/1956] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:41:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:41:40.408617 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52924ff5-b1f7-4520-80d3-957a3cd03b0c None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:40.411101 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-52924ff5-b1f7-4520-80d3-957a3cd03b0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:40.475670 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52924ff5-b1f7-4520-80d3-957a3cd03b0c None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:40.493350 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-ae9ff7d3-477b-4216-a829-a8cdfa9e508d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87571c73f1b442b58d174fc3b6adc8b5', 'request_id': 'req-ae9ff7d3-477b-4216-a829-a8cdfa9e508d'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:40.501442 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-ae9ff7d3-477b-4216-a829-a8cdfa9e508d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87571c73f1b442b58d174fc3b6adc8b5', 'request_id': 'req-ae9ff7d3-477b-4216-a829-a8cdfa9e508d'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:40.513216 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-ae9ff7d3-477b-4216-a829-a8cdfa9e508d admin admin] The token cache is being invalidate because project 87571c73f1b442b58d174fc3b6adc8b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81723) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:41:40.513216 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-ae9ff7d3-477b-4216-a829-a8cdfa9e508d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:40.513917 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-ae9ff7d3-477b-4216-a829-a8cdfa9e508d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:40.515830 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 982/1957] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:41:40 2022] DELETE /identity/v3/projects/87571c73f1b442b58d174fc3b6adc8b5 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:41:40.516536 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9ff7d3-477b-4216-a829-a8cdfa9e508d admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:40.518068 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-ae9ff7d3-477b-4216-a829-a8cdfa9e508d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:40.551914 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d19835d-fae1-40d5-a4ff-d556fdaa64a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36639dce4f97412f89d71367f650b901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:40.552693 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d19835d-fae1-40d5-a4ff-d556fdaa64a5 service cinder] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:40.552976 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d19835d-fae1-40d5-a4ff-d556fdaa64a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:40.553279 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d19835d-fae1-40d5-a4ff-d556fdaa64a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:40.555433 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-3d19835d-fae1-40d5-a4ff-d556fdaa64a5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:40.612837 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9ff7d3-477b-4216-a829-a8cdfa9e508d admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:40.624037 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c40a012-b71e-4388-ab79-a7103dfba829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:40.626345 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c40a012-b71e-4388-ab79-a7103dfba829 admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:40.626831 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c40a012-b71e-4388-ab79-a7103dfba829 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:40.627245 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c40a012-b71e-4388-ab79-a7103dfba829 admin admin] PATH_INFO: `/v3/projects` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:40.629326 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c40a012-b71e-4388-ab79-a7103dfba829 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:40.633108 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c40a012-b71e-4388-ab79-a7103dfba829 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:40.674522 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d19835d-fae1-40d5-a4ff-d556fdaa64a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:40.678413 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d19835d-fae1-40d5-a4ff-d556fdaa64a5 service cinder] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:40.697046 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 976/1958] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:41:40 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:41:40.698686 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d19835d-fae1-40d5-a4ff-d556fdaa64a5 service cinder] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:40.700864 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-3d19835d-fae1-40d5-a4ff-d556fdaa64a5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:40.726314 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d19835d-fae1-40d5-a4ff-d556fdaa64a5 service cinder] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:40.733382 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f3aec1-7a76-465a-adaf-3a4e89c1219f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:40.746635 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f3aec1-7a76-465a-adaf-3a4e89c1219f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:40.747597 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f3aec1-7a76-465a-adaf-3a4e89c1219f admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:40.747881 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f3aec1-7a76-465a-adaf-3a4e89c1219f admin admin] PATH_INFO: `/v3/users/94b65585ecf445d388d0c30a01ff8777` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:40.751747 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f3aec1-7a76-465a-adaf-3a4e89c1219f admin admin] RBAC: Authorizing `identity:delete_user(user_id=94b65585ecf445d388d0c30a01ff8777)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:40.754355 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f3aec1-7a76-465a-adaf-3a4e89c1219f admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:40.790888 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 983/1959] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:41:40 2022] POST /identity/v3/projects => generated 377 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:41:40.795611 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c40a012-b71e-4388-ab79-a7103dfba829 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:40.903033 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c40a012-b71e-4388-ab79-a7103dfba829 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:40.909742 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208f4e53-8dba-4ae9-b4e2-cd15a738898b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:40.918896 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208f4e53-8dba-4ae9-b4e2-cd15a738898b admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:40.919399 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208f4e53-8dba-4ae9-b4e2-cd15a738898b admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:40.919747 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208f4e53-8dba-4ae9-b4e2-cd15a738898b admin admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:40.920722 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208f4e53-8dba-4ae9-b4e2-cd15a738898b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:40.923446 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208f4e53-8dba-4ae9-b4e2-cd15a738898b admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:41.063662 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 984/1960] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:41:40 2022] POST /identity/v3/users => generated 289 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:41:41.063662 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208f4e53-8dba-4ae9-b4e2-cd15a738898b admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:41.065599 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-208f4e53-8dba-4ae9-b4e2-cd15a738898b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:41.076423 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-08f3aec1-7a76-465a-adaf-3a4e89c1219f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94b65585ecf445d388d0c30a01ff8777'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:41.077491 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-08f3aec1-7a76-465a-adaf-3a4e89c1219f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94b65585ecf445d388d0c30a01ff8777'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:41.078174 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-08f3aec1-7a76-465a-adaf-3a4e89c1219f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94b65585ecf445d388d0c30a01ff8777'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:41.087025 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-08f3aec1-7a76-465a-adaf-3a4e89c1219f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94b65585ecf445d388d0c30a01ff8777'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:41.139595 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208f4e53-8dba-4ae9-b4e2-cd15a738898b admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:41.139595 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c58dd3-43ba-4cdc-9f46-8e85ce70e359 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36639dce4f97412f89d71367f650b901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:41.139595 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c58dd3-43ba-4cdc-9f46-8e85ce70e359 service cinder] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:41.139595 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c58dd3-43ba-4cdc-9f46-8e85ce70e359 service cinder] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:41.140228 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c58dd3-43ba-4cdc-9f46-8e85ce70e359 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:41.141440 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-b5c58dd3-43ba-4cdc-9f46-8e85ce70e359 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:41.192981 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-08f3aec1-7a76-465a-adaf-3a4e89c1219f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94b65585ecf445d388d0c30a01ff8777'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:41.219817 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 977/1961] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:41:40 2022] DELETE /identity/v3/users/94b65585ecf445d388d0c30a01ff8777 => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:41:41.220705 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f3aec1-7a76-465a-adaf-3a4e89c1219f admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:41.222113 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-08f3aec1-7a76-465a-adaf-3a4e89c1219f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:41.222756 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c58dd3-43ba-4cdc-9f46-8e85ce70e359 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:41.224386 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c58dd3-43ba-4cdc-9f46-8e85ce70e359 service cinder] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:41.236899 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 985/1962] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:41:41 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:41:41.338124 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f3aec1-7a76-465a-adaf-3a4e89c1219f admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:41.343976 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eec6634-6ccb-49de-b71b-e25ae87260d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:41.344695 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eec6634-6ccb-49de-b71b-e25ae87260d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:41.345204 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eec6634-6ccb-49de-b71b-e25ae87260d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:41.345516 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eec6634-6ccb-49de-b71b-e25ae87260d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:41.346232 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eec6634-6ccb-49de-b71b-e25ae87260d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:41.347910 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eec6634-6ccb-49de-b71b-e25ae87260d6 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:41.360534 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 978/1963] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:41:41 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:41:41.364647 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c58dd3-43ba-4cdc-9f46-8e85ce70e359 service cinder] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:41.375618 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c58dd3-43ba-4cdc-9f46-8e85ce70e359 service cinder] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:41.380312 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dc8e3a-4aae-4192-91b7-0b2a9c6c8d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:41.382398 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dc8e3a-4aae-4192-91b7-0b2a9c6c8d39 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:41.382398 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dc8e3a-4aae-4192-91b7-0b2a9c6c8d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:41.382398 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dc8e3a-4aae-4192-91b7-0b2a9c6c8d39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:41.382398 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dc8e3a-4aae-4192-91b7-0b2a9c6c8d39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:41.387804 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dc8e3a-4aae-4192-91b7-0b2a9c6c8d39 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:41.403282 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 986/1964] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:41:41 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:41:41.407250 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eec6634-6ccb-49de-b71b-e25ae87260d6 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:41.418245 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eec6634-6ccb-49de-b71b-e25ae87260d6 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:41.423436 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc45f35-7e84-42f2-84cf-a3230c5bcd97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:41.424127 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc45f35-7e84-42f2-84cf-a3230c5bcd97 admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:41.424413 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc45f35-7e84-42f2-84cf-a3230c5bcd97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:41.424707 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc45f35-7e84-42f2-84cf-a3230c5bcd97 admin admin] PATH_INFO: `/v3/projects/d217f2dbe6c6409784d1275e269fd6b8/users/591b523735cc4f64a2215d634810eb5b/roles/e1f9b167c51c4a388cbc9594db8a465d` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:41.441829 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc45f35-7e84-42f2-84cf-a3230c5bcd97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d217f2dbe6c6409784d1275e269fd6b8, user_id=591b523735cc4f64a2215d634810eb5b, role_id=e1f9b167c51c4a388cbc9594db8a465d)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:41.444564 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc45f35-7e84-42f2-84cf-a3230c5bcd97 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:41.572332 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 979/1965] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:41:41 2022] PUT /identity/v3/projects/d217f2dbe6c6409784d1275e269fd6b8/users/591b523735cc4f64a2215d634810eb5b/roles/e1f9b167c51c4a388cbc9594db8a465d => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:41:41.585137 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d88053-a8f1-47b6-b8ee-a7eb1f48ebfc None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:41.585502 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d88053-a8f1-47b6-b8ee-a7eb1f48ebfc None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:41.585730 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d88053-a8f1-47b6-b8ee-a7eb1f48ebfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:41.617748 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-a4d88053-a8f1-47b6-b8ee-a7eb1f48ebfc None None] MFA Rules not processed for user `591b523735cc4f64a2215d634810eb5b`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:41:41.618983 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-a4d88053-a8f1-47b6-b8ee-a7eb1f48ebfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:41.731576 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 987/1966] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:41:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:41:41.736261 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc45f35-7e84-42f2-84cf-a3230c5bcd97 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:41.805971 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc45f35-7e84-42f2-84cf-a3230c5bcd97 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:41.815415 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b9fdcd-40df-4e34-aafa-50fbea9df3b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:41.816487 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b9fdcd-40df-4e34-aafa-50fbea9df3b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:41.816958 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b9fdcd-40df-4e34-aafa-50fbea9df3b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:41.817522 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b9fdcd-40df-4e34-aafa-50fbea9df3b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:41.818621 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b9fdcd-40df-4e34-aafa-50fbea9df3b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:41.820214 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d88053-a8f1-47b6-b8ee-a7eb1f48ebfc None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:41.822453 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-a4d88053-a8f1-47b6-b8ee-a7eb1f48ebfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:41.823137 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b9fdcd-40df-4e34-aafa-50fbea9df3b6 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:41.838226 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d88053-a8f1-47b6-b8ee-a7eb1f48ebfc None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:41.844659 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3738f2-1941-418c-bfdb-5c73d47ba5ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:41.845924 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3738f2-1941-418c-bfdb-5c73d47ba5ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:41.845924 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3738f2-1941-418c-bfdb-5c73d47ba5ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:41.846113 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3738f2-1941-418c-bfdb-5c73d47ba5ff admin admin] PATH_INFO: `/v3/projects/f77f41d338954547bd204f0c65a0e9ef` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:41.847453 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3738f2-1941-418c-bfdb-5c73d47ba5ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=f77f41d338954547bd204f0c65a0e9ef)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:41.849777 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3738f2-1941-418c-bfdb-5c73d47ba5ff admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:41.971247 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 980/1967] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:41:41 2022] POST /identity/v3/projects => generated 377 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:41:41.973817 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b9fdcd-40df-4e34-aafa-50fbea9df3b6 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:42.045461 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b9fdcd-40df-4e34-aafa-50fbea9df3b6 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:42.050515 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee9345e-7209-42d5-ba9f-74c2c6d9b8f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:42.051192 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee9345e-7209-42d5-ba9f-74c2c6d9b8f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:42.051480 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee9345e-7209-42d5-ba9f-74c2c6d9b8f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:42.051824 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee9345e-7209-42d5-ba9f-74c2c6d9b8f2 admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:42.054098 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee9345e-7209-42d5-ba9f-74c2c6d9b8f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:42.057257 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee9345e-7209-42d5-ba9f-74c2c6d9b8f2 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:42.070400 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-fb3738f2-1941-418c-bfdb-5c73d47ba5ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f77f41d338954547bd204f0c65a0e9ef', 'request_id': 'req-fb3738f2-1941-418c-bfdb-5c73d47ba5ff'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:42.177573 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 981/1968] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:41:41 2022] POST /identity/v3/users => generated 295 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:41:42.178878 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-fb3738f2-1941-418c-bfdb-5c73d47ba5ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f77f41d338954547bd204f0c65a0e9ef', 'request_id': 'req-fb3738f2-1941-418c-bfdb-5c73d47ba5ff'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:42.183363 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee9345e-7209-42d5-ba9f-74c2c6d9b8f2 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:42.189225 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-fb3738f2-1941-418c-bfdb-5c73d47ba5ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f77f41d338954547bd204f0c65a0e9ef', 'request_id': 'req-fb3738f2-1941-418c-bfdb-5c73d47ba5ff'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:42.195536 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee9345e-7209-42d5-ba9f-74c2c6d9b8f2 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:42.202863 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-fb3738f2-1941-418c-bfdb-5c73d47ba5ff admin admin] The token cache is being invalidate because project f77f41d338954547bd204f0c65a0e9ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81723) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:41:42.203275 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-fb3738f2-1941-418c-bfdb-5c73d47ba5ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:42.204051 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-fb3738f2-1941-418c-bfdb-5c73d47ba5ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:42.206415 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 988/1969] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:41:41 2022] DELETE /identity/v3/projects/f77f41d338954547bd204f0c65a0e9ef => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:41:42.211625 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c25e4d-9e14-47bf-92e8-75b92cfa2b06 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:42.212042 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c25e4d-9e14-47bf-92e8-75b92cfa2b06 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:42.212413 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c25e4d-9e14-47bf-92e8-75b92cfa2b06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:42.242884 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-13c25e4d-9e14-47bf-92e8-75b92cfa2b06 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:41:42.244322 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-13c25e4d-9e14-47bf-92e8-75b92cfa2b06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:42.273027 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2428623b-ce34-4c7c-934b-cc51352d01fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:42.273717 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2428623b-ce34-4c7c-934b-cc51352d01fc admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:42.274001 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2428623b-ce34-4c7c-934b-cc51352d01fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:42.274957 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2428623b-ce34-4c7c-934b-cc51352d01fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:42.274957 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2428623b-ce34-4c7c-934b-cc51352d01fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:42.276561 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2428623b-ce34-4c7c-934b-cc51352d01fc admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:42.289861 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 982/1970] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:41:42 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:41:42.295740 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2428623b-ce34-4c7c-934b-cc51352d01fc admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:42.297940 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-2428623b-ce34-4c7c-934b-cc51352d01fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:42.314578 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2428623b-ce34-4c7c-934b-cc51352d01fc admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:42.342591 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e612e7f9-1ddd-4c1f-8349-6ba09ec9f324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:42.342757 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 989/1971] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:41:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:41:42.349797 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e612e7f9-1ddd-4c1f-8349-6ba09ec9f324 admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:42.350332 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e612e7f9-1ddd-4c1f-8349-6ba09ec9f324 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:42.350689 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e612e7f9-1ddd-4c1f-8349-6ba09ec9f324 admin admin] PATH_INFO: `/v3/projects/8e61949114b44623894c127dd51a3efc/users/9b94e803b36144d8a255e3566e612060/roles/486144ad1bcd417e86acdbc1d14327bf` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:42.351878 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6426d1-e2bd-41fb-94e7-4644fb9475e7 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:42.352460 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6426d1-e2bd-41fb-94e7-4644fb9475e7 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:42.352813 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6426d1-e2bd-41fb-94e7-4644fb9475e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:42.385593 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e612e7f9-1ddd-4c1f-8349-6ba09ec9f324 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e61949114b44623894c127dd51a3efc, user_id=9b94e803b36144d8a255e3566e612060, role_id=486144ad1bcd417e86acdbc1d14327bf)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:42.388582 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-8d6426d1-e2bd-41fb-94e7-4644fb9475e7 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:41:42.390411 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-8d6426d1-e2bd-41fb-94e7-4644fb9475e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:42.391057 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e612e7f9-1ddd-4c1f-8349-6ba09ec9f324 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:42.417849 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 990/1972] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:41:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:41:42.427738 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6426d1-e2bd-41fb-94e7-4644fb9475e7 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:42.448499 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6426d1-e2bd-41fb-94e7-4644fb9475e7 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:42.458462 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c898b04-00e6-4e30-917a-c70fd0f2c9d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:42.459097 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c898b04-00e6-4e30-917a-c70fd0f2c9d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:42.459383 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c898b04-00e6-4e30-917a-c70fd0f2c9d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:42.459662 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c898b04-00e6-4e30-917a-c70fd0f2c9d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:42.460447 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c898b04-00e6-4e30-917a-c70fd0f2c9d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:41:42.461648 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c898b04-00e6-4e30-917a-c70fd0f2c9d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:42.463954 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c898b04-00e6-4e30-917a-c70fd0f2c9d2 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:42.485437 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 991/1973] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 24 06:41:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:41:42.490389 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c898b04-00e6-4e30-917a-c70fd0f2c9d2 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:42.507932 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c898b04-00e6-4e30-917a-c70fd0f2c9d2 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:42.515777 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b8ea93-808d-4447-b772-1fb030f1cef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:42.516522 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b8ea93-808d-4447-b772-1fb030f1cef3 admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:42.516908 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b8ea93-808d-4447-b772-1fb030f1cef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:42.517324 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b8ea93-808d-4447-b772-1fb030f1cef3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:42.518031 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b8ea93-808d-4447-b772-1fb030f1cef3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:42.520300 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b8ea93-808d-4447-b772-1fb030f1cef3 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:42.584770 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 983/1974] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:41:42 2022] PUT /identity/v3/projects/8e61949114b44623894c127dd51a3efc/users/9b94e803b36144d8a255e3566e612060/roles/486144ad1bcd417e86acdbc1d14327bf => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:41:42.586766 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e612e7f9-1ddd-4c1f-8349-6ba09ec9f324 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:42.682443 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e612e7f9-1ddd-4c1f-8349-6ba09ec9f324 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:42.683062 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 992/1975] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:41:42 2022] POST /identity/v3/projects => generated 367 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:41:42.688617 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b8ea93-808d-4447-b772-1fb030f1cef3 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:42.755192 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5337257f-af11-440b-9adc-d3bd7b389b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:42.756131 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5337257f-af11-440b-9adc-d3bd7b389b92 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:42.756541 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5337257f-af11-440b-9adc-d3bd7b389b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:42.756954 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5337257f-af11-440b-9adc-d3bd7b389b92 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:42.758042 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5337257f-af11-440b-9adc-d3bd7b389b92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:42.759990 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5337257f-af11-440b-9adc-d3bd7b389b92 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:42.770812 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b8ea93-808d-4447-b772-1fb030f1cef3 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:42.772813 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 984/1976] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:41:42 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:41:42.777882 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5337257f-af11-440b-9adc-d3bd7b389b92 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:42.780595 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f495cf3-c348-4773-a359-ee0147075b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:42.781256 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f495cf3-c348-4773-a359-ee0147075b5b admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:42.781544 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f495cf3-c348-4773-a359-ee0147075b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:42.781827 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f495cf3-c348-4773-a359-ee0147075b5b admin admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:42.782472 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f495cf3-c348-4773-a359-ee0147075b5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:42.784617 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f495cf3-c348-4773-a359-ee0147075b5b admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:42.814735 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5337257f-af11-440b-9adc-d3bd7b389b92 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:42.820456 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e15c834-dfaa-45b0-856c-3bb42547f665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:42.821647 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e15c834-dfaa-45b0-856c-3bb42547f665 admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:42.822344 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e15c834-dfaa-45b0-856c-3bb42547f665 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:42.823367 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e15c834-dfaa-45b0-856c-3bb42547f665 admin admin] PATH_INFO: `/v3/projects/8e61949114b44623894c127dd51a3efc/users/9b94e803b36144d8a255e3566e612060/roles/486144ad1bcd417e86acdbc1d14327bf` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:42.827575 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e15c834-dfaa-45b0-856c-3bb42547f665 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e61949114b44623894c127dd51a3efc, user_id=9b94e803b36144d8a255e3566e612060, role_id=486144ad1bcd417e86acdbc1d14327bf)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:42.831517 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e15c834-dfaa-45b0-856c-3bb42547f665 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:42.860165 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 985/1977] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:41:42 2022] PUT /identity/v3/projects/8e61949114b44623894c127dd51a3efc/users/9b94e803b36144d8a255e3566e612060/roles/486144ad1bcd417e86acdbc1d14327bf => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:41:42.866639 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa7e9f4-1657-4322-8ed9-c66144c7fb7c None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:42.866937 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa7e9f4-1657-4322-8ed9-c66144c7fb7c None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:42.867219 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa7e9f4-1657-4322-8ed9-c66144c7fb7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:42.910052 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-8aa7e9f4-1657-4322-8ed9-c66144c7fb7c None None] MFA Rules not processed for user `9b94e803b36144d8a255e3566e612060`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:41:42.911433 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-8aa7e9f4-1657-4322-8ed9-c66144c7fb7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:42.952866 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 993/1978] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:41:42 2022] POST /identity/v3/users => generated 284 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:41:42.958449 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f495cf3-c348-4773-a359-ee0147075b5b admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:42.979848 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f495cf3-c348-4773-a359-ee0147075b5b admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:43.030591 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 986/1979] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:41:42 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:41:43.050225 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa7e9f4-1657-4322-8ed9-c66144c7fb7c None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:43.052556 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-8aa7e9f4-1657-4322-8ed9-c66144c7fb7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:43.066261 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a67169-9c15-4702-a837-39e6604034e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:43.069064 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a67169-9c15-4702-a837-39e6604034e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:43.070039 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a67169-9c15-4702-a837-39e6604034e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:43.070919 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a67169-9c15-4702-a837-39e6604034e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:43.072041 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a67169-9c15-4702-a837-39e6604034e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:43.073729 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a67169-9c15-4702-a837-39e6604034e0 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:43.085360 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 994/1980] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:41:42 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:41:43.089697 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a67169-9c15-4702-a837-39e6604034e0 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:43.101311 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a67169-9c15-4702-a837-39e6604034e0 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:43.106124 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f3708d-ada7-46de-9890-a1ea4b61038d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:43.106732 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f3708d-ada7-46de-9890-a1ea4b61038d admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:43.107183 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f3708d-ada7-46de-9890-a1ea4b61038d admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:43.107782 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f3708d-ada7-46de-9890-a1ea4b61038d admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:43.108045 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f3708d-ada7-46de-9890-a1ea4b61038d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:43.109701 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f3708d-ada7-46de-9890-a1ea4b61038d admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:43.121594 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 995/1981] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:41:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:41:43.124071 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa7e9f4-1657-4322-8ed9-c66144c7fb7c None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:43.125592 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f3708d-ada7-46de-9890-a1ea4b61038d admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:43.127890 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286aa541-e2ac-4843-ab20-51a18a096f79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36639dce4f97412f89d71367f650b901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:43.128523 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286aa541-e2ac-4843-ab20-51a18a096f79 service cinder] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:43.128800 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286aa541-e2ac-4843-ab20-51a18a096f79 service cinder] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:43.129075 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286aa541-e2ac-4843-ab20-51a18a096f79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:43.131342 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-286aa541-e2ac-4843-ab20-51a18a096f79 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:43.139925 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f3708d-ada7-46de-9890-a1ea4b61038d admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:43.143093 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f957091-77b5-46de-9f8f-15172269bc8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:43.143864 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f957091-77b5-46de-9f8f-15172269bc8d admin admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:43.144271 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f957091-77b5-46de-9f8f-15172269bc8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:43.144562 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f957091-77b5-46de-9f8f-15172269bc8d admin admin] PATH_INFO: `/v3/projects/9f9adde8befc4c25a2fde3c26622ce83/users/b5d6c36fb83c4955849c98cb9f96a6aa/roles/e1f9b167c51c4a388cbc9594db8a465d` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:43.162313 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f957091-77b5-46de-9f8f-15172269bc8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f9adde8befc4c25a2fde3c26622ce83, user_id=b5d6c36fb83c4955849c98cb9f96a6aa, role_id=e1f9b167c51c4a388cbc9594db8a465d)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:43.165680 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f957091-77b5-46de-9f8f-15172269bc8d admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:43.197219 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286aa541-e2ac-4843-ab20-51a18a096f79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:43.198667 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286aa541-e2ac-4843-ab20-51a18a096f79 service cinder] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:43.210346 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 987/1982] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:41:43 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:41:43.211668 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286aa541-e2ac-4843-ab20-51a18a096f79 service cinder] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:43.213277 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-286aa541-e2ac-4843-ab20-51a18a096f79 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:43.226339 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286aa541-e2ac-4843-ab20-51a18a096f79 service cinder] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:43.230201 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b564319b-c50c-437d-8ca0-25dea5a85bb2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36639dce4f97412f89d71367f650b901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:43.231763 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b564319b-c50c-437d-8ca0-25dea5a85bb2 service cinder] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:43.232579 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b564319b-c50c-437d-8ca0-25dea5a85bb2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:43.232579 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b564319b-c50c-437d-8ca0-25dea5a85bb2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:43.234735 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-b564319b-c50c-437d-8ca0-25dea5a85bb2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:43.297120 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 996/1983] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:41:43 2022] PUT /identity/v3/projects/9f9adde8befc4c25a2fde3c26622ce83/users/b5d6c36fb83c4955849c98cb9f96a6aa/roles/e1f9b167c51c4a388cbc9594db8a465d => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:41:43.299669 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b564319b-c50c-437d-8ca0-25dea5a85bb2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:43.301201 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b564319b-c50c-437d-8ca0-25dea5a85bb2 service cinder] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:43.309842 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14d2fa0-9230-429c-b9fa-d04cef89ff57 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:43.310352 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14d2fa0-9230-429c-b9fa-d04cef89ff57 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:43.310759 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14d2fa0-9230-429c-b9fa-d04cef89ff57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:43.311959 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 988/1984] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:41:43 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:41:43.357597 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-e14d2fa0-9230-429c-b9fa-d04cef89ff57 None None] MFA Rules not processed for user `b5d6c36fb83c4955849c98cb9f96a6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:41:43.358876 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-e14d2fa0-9230-429c-b9fa-d04cef89ff57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:43.394285 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:41:43.442233 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 997/1985] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:41:43 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:41:43.448546 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b564319b-c50c-437d-8ca0-25dea5a85bb2 service cinder] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:43.477116 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b564319b-c50c-437d-8ca0-25dea5a85bb2 service cinder] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:43.550979 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c87dc5-775f-4e1a-b15b-4e4ca3a38f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:43.551684 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c87dc5-775f-4e1a-b15b-4e4ca3a38f6c admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:43.551956 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c87dc5-775f-4e1a-b15b-4e4ca3a38f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:43.552222 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c87dc5-775f-4e1a-b15b-4e4ca3a38f6c admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:43.552855 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c87dc5-775f-4e1a-b15b-4e4ca3a38f6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:43.554826 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c87dc5-775f-4e1a-b15b-4e4ca3a38f6c admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:43.765337 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 989/1986] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:41:43 2022] POST /identity/v3/projects => generated 367 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:41:43.770650 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14d2fa0-9230-429c-b9fa-d04cef89ff57 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:43.788618 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14d2fa0-9230-429c-b9fa-d04cef89ff57 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:43.897361 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945358b4-10bf-4c4d-a715-6fffd23455bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:43.898771 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945358b4-10bf-4c4d-a715-6fffd23455bc admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:43.899324 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945358b4-10bf-4c4d-a715-6fffd23455bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:43.899771 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945358b4-10bf-4c4d-a715-6fffd23455bc admin admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:43.900759 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945358b4-10bf-4c4d-a715-6fffd23455bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:43.905270 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945358b4-10bf-4c4d-a715-6fffd23455bc admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:43.947750 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:41:44.093848 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 998/1987] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:41:43 2022] POST /identity/v3/users => generated 290 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:41:44.094868 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c87dc5-775f-4e1a-b15b-4e4ca3a38f6c admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:44.127240 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c87dc5-775f-4e1a-b15b-4e4ca3a38f6c admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:44.137173 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fd630b-2820-45a7-8402-a06f4603a71c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:44.139091 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fd630b-2820-45a7-8402-a06f4603a71c admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:44.139091 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fd630b-2820-45a7-8402-a06f4603a71c admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:44.139091 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fd630b-2820-45a7-8402-a06f4603a71c admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:44.139091 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fd630b-2820-45a7-8402-a06f4603a71c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:44.141601 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fd630b-2820-45a7-8402-a06f4603a71c admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:44.198894 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 990/1988] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:41:44 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:41:44.200168 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945358b4-10bf-4c4d-a715-6fffd23455bc admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:44.200168 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945358b4-10bf-4c4d-a715-6fffd23455bc admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:44.200168 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b5d2b8-291f-40ab-a58c-15f9bbee11b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:44.200168 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b5d2b8-291f-40ab-a58c-15f9bbee11b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:44.200551 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b5d2b8-291f-40ab-a58c-15f9bbee11b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:44.200551 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b5d2b8-291f-40ab-a58c-15f9bbee11b4 admin admin] PATH_INFO: `/v3/projects/da369e0dd1eb43cab32ea0cd064a28d4/users/9120eb33e6904c5c845a3c59448bc551/roles/486144ad1bcd417e86acdbc1d14327bf` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:44.221536 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b5d2b8-291f-40ab-a58c-15f9bbee11b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da369e0dd1eb43cab32ea0cd064a28d4, user_id=9120eb33e6904c5c845a3c59448bc551, role_id=486144ad1bcd417e86acdbc1d14327bf)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:44.224076 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b5d2b8-291f-40ab-a58c-15f9bbee11b4 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:44.573963 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 999/1989] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:41:44 2022] PUT /identity/v3/projects/da369e0dd1eb43cab32ea0cd064a28d4/users/9120eb33e6904c5c845a3c59448bc551/roles/486144ad1bcd417e86acdbc1d14327bf => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:41:44.576307 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fd630b-2820-45a7-8402-a06f4603a71c admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:44.601099 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fd630b-2820-45a7-8402-a06f4603a71c admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:44.754060 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74d5f75-194a-4664-a5b8-3fe7815d7513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:44.755109 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74d5f75-194a-4664-a5b8-3fe7815d7513 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:44.755610 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74d5f75-194a-4664-a5b8-3fe7815d7513 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:44.756093 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74d5f75-194a-4664-a5b8-3fe7815d7513 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:44.763842 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74d5f75-194a-4664-a5b8-3fe7815d7513 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:44.767115 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74d5f75-194a-4664-a5b8-3fe7815d7513 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:44.784234 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 991/1990] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:41:44 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:41:44.788041 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b5d2b8-291f-40ab-a58c-15f9bbee11b4 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:44.833798 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b5d2b8-291f-40ab-a58c-15f9bbee11b4 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:44.850308 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d8bd36-cdd7-4b59-a1a8-689a55ef1b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:44.851070 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d8bd36-cdd7-4b59-a1a8-689a55ef1b32 admin admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:44.851483 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d8bd36-cdd7-4b59-a1a8-689a55ef1b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:44.851785 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d8bd36-cdd7-4b59-a1a8-689a55ef1b32 admin admin] PATH_INFO: `/v3/projects/da369e0dd1eb43cab32ea0cd064a28d4/users/9120eb33e6904c5c845a3c59448bc551/roles/486144ad1bcd417e86acdbc1d14327bf` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:44.872720 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d8bd36-cdd7-4b59-a1a8-689a55ef1b32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da369e0dd1eb43cab32ea0cd064a28d4, user_id=9120eb33e6904c5c845a3c59448bc551, role_id=486144ad1bcd417e86acdbc1d14327bf)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:44.879172 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d8bd36-cdd7-4b59-a1a8-689a55ef1b32 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:44.901048 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1000/1991] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:41:44 2022] PUT /identity/v3/projects/da369e0dd1eb43cab32ea0cd064a28d4/users/9120eb33e6904c5c845a3c59448bc551/roles/486144ad1bcd417e86acdbc1d14327bf => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:41:44.926371 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0842766-6f94-4626-84f2-2e055bf1d650 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:44.927037 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0842766-6f94-4626-84f2-2e055bf1d650 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:44.927534 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0842766-6f94-4626-84f2-2e055bf1d650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:45.010202 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-c0842766-6f94-4626-84f2-2e055bf1d650 None None] MFA Rules not processed for user `9120eb33e6904c5c845a3c59448bc551`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:41:45.011762 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-c0842766-6f94-4626-84f2-2e055bf1d650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:45.165843 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 992/1992] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:41:44 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:41:45.593706 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d8bd36-cdd7-4b59-a1a8-689a55ef1b32 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:45.700664 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d8bd36-cdd7-4b59-a1a8-689a55ef1b32 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:45.704682 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:41:45.706959 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a746fd7-abc5-47e1-b245-4bab3bd46bfc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36639dce4f97412f89d71367f650b901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:45.708032 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a746fd7-abc5-47e1-b245-4bab3bd46bfc service cinder] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:45.708619 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a746fd7-abc5-47e1-b245-4bab3bd46bfc service cinder] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:45.709432 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a746fd7-abc5-47e1-b245-4bab3bd46bfc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:45.760034 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a746fd7-abc5-47e1-b245-4bab3bd46bfc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:45.773836 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a746fd7-abc5-47e1-b245-4bab3bd46bfc service cinder] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:45.803392 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1001/1993] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:41:45 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:41:46.182544 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0842766-6f94-4626-84f2-2e055bf1d650 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:46.194355 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0842766-6f94-4626-84f2-2e055bf1d650 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:46.210613 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392a80ff-04a4-44d2-bb7a-0d34ed9285ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36639dce4f97412f89d71367f650b901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:46.210613 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392a80ff-04a4-44d2-bb7a-0d34ed9285ff service cinder] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:46.210613 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392a80ff-04a4-44d2-bb7a-0d34ed9285ff service cinder] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:46.210613 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392a80ff-04a4-44d2-bb7a-0d34ed9285ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:46.326856 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392a80ff-04a4-44d2-bb7a-0d34ed9285ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:46.357101 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392a80ff-04a4-44d2-bb7a-0d34ed9285ff service cinder] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:46.373433 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 993/1994] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:41:46 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:41:46.907057 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=86,keystone:DELETE=9,keystone:INSERT=5 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:41:47.275848 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:41:47.638177 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a746fd7-abc5-47e1-b245-4bab3bd46bfc service cinder] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:47.656239 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a746fd7-abc5-47e1-b245-4bab3bd46bfc service cinder] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:47.663398 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718eb43a-44a1-4976-8fae-874141f92622 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36639dce4f97412f89d71367f650b901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:47.664728 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718eb43a-44a1-4976-8fae-874141f92622 service cinder] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:47.665107 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718eb43a-44a1-4976-8fae-874141f92622 service cinder] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:47.665603 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718eb43a-44a1-4976-8fae-874141f92622 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:47.669121 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-718eb43a-44a1-4976-8fae-874141f92622 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:47.797887 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718eb43a-44a1-4976-8fae-874141f92622 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:47.799556 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718eb43a-44a1-4976-8fae-874141f92622 service cinder] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:47.818074 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1002/1995] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:41:47 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:41:49.660716 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:41:50.283523 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:41:51.199976 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392a80ff-04a4-44d2-bb7a-0d34ed9285ff service cinder] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:51.201623 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-392a80ff-04a4-44d2-bb7a-0d34ed9285ff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:51.221440 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718eb43a-44a1-4976-8fae-874141f92622 service cinder] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:51.233369 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718eb43a-44a1-4976-8fae-874141f92622 service cinder] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:51.238350 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27c98a3-d4be-4cbf-a2a1-a83321dd9798 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36639dce4f97412f89d71367f650b901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:51.242155 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27c98a3-d4be-4cbf-a2a1-a83321dd9798 service cinder] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:51.242155 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27c98a3-d4be-4cbf-a2a1-a83321dd9798 service cinder] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:51.242155 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27c98a3-d4be-4cbf-a2a1-a83321dd9798 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:51.330826 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392a80ff-04a4-44d2-bb7a-0d34ed9285ff service cinder] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:51.336569 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad7c4cc-d582-4bcf-86ae-795b0e1536d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:51.337639 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad7c4cc-d582-4bcf-86ae-795b0e1536d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:51.338147 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad7c4cc-d582-4bcf-86ae-795b0e1536d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:51.338533 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad7c4cc-d582-4bcf-86ae-795b0e1536d7 admin admin] PATH_INFO: `/v3/users/cb65b909126b42af88786fa4e6c793a1` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:51.340887 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad7c4cc-d582-4bcf-86ae-795b0e1536d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb65b909126b42af88786fa4e6c793a1)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:51.343735 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad7c4cc-d582-4bcf-86ae-795b0e1536d7 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:51.406775 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27c98a3-d4be-4cbf-a2a1-a83321dd9798 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:51.409410 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27c98a3-d4be-4cbf-a2a1-a83321dd9798 service cinder] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:51.446519 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1003/1996] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:41:51 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:41:51.527650 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27c98a3-d4be-4cbf-a2a1-a83321dd9798 service cinder] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:51.529261 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-c27c98a3-d4be-4cbf-a2a1-a83321dd9798 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:51.544837 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27c98a3-d4be-4cbf-a2a1-a83321dd9798 service cinder] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:51.552801 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b038ec-32f7-42c2-8e6d-746745d512f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:51.552801 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b038ec-32f7-42c2-8e6d-746745d512f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:51.552801 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b038ec-32f7-42c2-8e6d-746745d512f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:51.552801 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b038ec-32f7-42c2-8e6d-746745d512f0 admin admin] PATH_INFO: `/v3/users/abde191473584a778b14bd77a06aa835` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:51.553799 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b038ec-32f7-42c2-8e6d-746745d512f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=abde191473584a778b14bd77a06aa835)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:51.555240 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b038ec-32f7-42c2-8e6d-746745d512f0 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:51.630728 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-dad7c4cc-d582-4bcf-86ae-795b0e1536d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb65b909126b42af88786fa4e6c793a1'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:51.631805 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-dad7c4cc-d582-4bcf-86ae-795b0e1536d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb65b909126b42af88786fa4e6c793a1'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:51.632595 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-dad7c4cc-d582-4bcf-86ae-795b0e1536d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb65b909126b42af88786fa4e6c793a1'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:51.648528 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-dad7c4cc-d582-4bcf-86ae-795b0e1536d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb65b909126b42af88786fa4e6c793a1'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:51.809005 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-dad7c4cc-d582-4bcf-86ae-795b0e1536d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb65b909126b42af88786fa4e6c793a1'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:51.844303 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 994/1997] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:41:51 2022] DELETE /identity/v3/users/cb65b909126b42af88786fa4e6c793a1 => generated 0 bytes in 645 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:41:51.860499 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835d6bc6-1db5-4b93-a235-7b2d6751ec22 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:51.860996 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835d6bc6-1db5-4b93-a235-7b2d6751ec22 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:51.861521 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835d6bc6-1db5-4b93-a235-7b2d6751ec22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:51.890390 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-835d6bc6-1db5-4b93-a235-7b2d6751ec22 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:41:51.892572 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-835d6bc6-1db5-4b93-a235-7b2d6751ec22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:51.938184 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:41:51.958385 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-41b038ec-32f7-42c2-8e6d-746745d512f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abde191473584a778b14bd77a06aa835'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:51.959419 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-41b038ec-32f7-42c2-8e6d-746745d512f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abde191473584a778b14bd77a06aa835'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:51.960347 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-41b038ec-32f7-42c2-8e6d-746745d512f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abde191473584a778b14bd77a06aa835'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:51.983790 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 995/1998] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:41:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:41:52.001285 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835d6bc6-1db5-4b93-a235-7b2d6751ec22 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:52.003085 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-835d6bc6-1db5-4b93-a235-7b2d6751ec22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:52.072527 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835d6bc6-1db5-4b93-a235-7b2d6751ec22 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:52.076836 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2f45f9-f093-40e9-a076-9183421b9593 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:52.077487 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2f45f9-f093-40e9-a076-9183421b9593 service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:52.077785 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2f45f9-f093-40e9-a076-9183421b9593 service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:52.078040 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2f45f9-f093-40e9-a076-9183421b9593 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:52.093913 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2f45f9-f093-40e9-a076-9183421b9593 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:52.093913 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2f45f9-f093-40e9-a076-9183421b9593 service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:52.101092 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 996/1999] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:41:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:41:52.149245 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-41b038ec-32f7-42c2-8e6d-746745d512f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abde191473584a778b14bd77a06aa835'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:52.168808 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-41b038ec-32f7-42c2-8e6d-746745d512f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abde191473584a778b14bd77a06aa835'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:52.194006 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1004/2000] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:41:51 2022] DELETE /identity/v3/users/abde191473584a778b14bd77a06aa835 => generated 0 bytes in 663 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:41:52.195593 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b443962-2a0b-413f-b08c-9c8caa1a11db None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:52.195891 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b443962-2a0b-413f-b08c-9c8caa1a11db None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:52.196348 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b443962-2a0b-413f-b08c-9c8caa1a11db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:52.227204 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-0b443962-2a0b-413f-b08c-9c8caa1a11db None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:41:52.228734 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-0b443962-2a0b-413f-b08c-9c8caa1a11db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:52.319472 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 997/2001] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:41:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:41:52.335824 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b038ec-32f7-42c2-8e6d-746745d512f0 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:52.421361 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b038ec-32f7-42c2-8e6d-746745d512f0 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:52.425394 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4db57a6-6dc8-4ea2-8e8d-6672b84907ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:52.426116 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4db57a6-6dc8-4ea2-8e8d-6672b84907ea service neutron] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:52.426397 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4db57a6-6dc8-4ea2-8e8d-6672b84907ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:52.426673 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4db57a6-6dc8-4ea2-8e8d-6672b84907ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:52.436633 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4db57a6-6dc8-4ea2-8e8d-6672b84907ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:52.438111 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4db57a6-6dc8-4ea2-8e8d-6672b84907ea service neutron] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:52.448479 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1005/2002] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:41:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:41:52.514764 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:41:52.619509 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b443962-2a0b-413f-b08c-9c8caa1a11db None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:52.626181 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-0b443962-2a0b-413f-b08c-9c8caa1a11db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:52.649245 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b443962-2a0b-413f-b08c-9c8caa1a11db None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:52.654858 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a71d30-ea0b-4a0c-986e-15a680ff1f4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:52.656807 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a71d30-ea0b-4a0c-986e-15a680ff1f4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:52.656807 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a71d30-ea0b-4a0c-986e-15a680ff1f4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:52.656807 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a71d30-ea0b-4a0c-986e-15a680ff1f4a admin admin] PATH_INFO: `/v3/projects/e676a17baa754a9fa4eb6ee91149c0b2` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:52.659033 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a71d30-ea0b-4a0c-986e-15a680ff1f4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e676a17baa754a9fa4eb6ee91149c0b2)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:52.661049 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a71d30-ea0b-4a0c-986e-15a680ff1f4a admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:52.827937 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-78a71d30-ea0b-4a0c-986e-15a680ff1f4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e676a17baa754a9fa4eb6ee91149c0b2', 'request_id': 'req-78a71d30-ea0b-4a0c-986e-15a680ff1f4a'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:52.835697 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-78a71d30-ea0b-4a0c-986e-15a680ff1f4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e676a17baa754a9fa4eb6ee91149c0b2', 'request_id': 'req-78a71d30-ea0b-4a0c-986e-15a680ff1f4a'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:53.055386 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-78a71d30-ea0b-4a0c-986e-15a680ff1f4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e676a17baa754a9fa4eb6ee91149c0b2', 'request_id': 'req-78a71d30-ea0b-4a0c-986e-15a680ff1f4a'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:53.058358 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4db57a6-6dc8-4ea2-8e8d-6672b84907ea service neutron] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:53.059872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-b4db57a6-6dc8-4ea2-8e8d-6672b84907ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:53.075203 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-78a71d30-ea0b-4a0c-986e-15a680ff1f4a admin admin] The token cache is being invalidate because project e676a17baa754a9fa4eb6ee91149c0b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:41:53.075203 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-78a71d30-ea0b-4a0c-986e-15a680ff1f4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:53.075203 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-78a71d30-ea0b-4a0c-986e-15a680ff1f4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:53.075203 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 998/2003] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:41:52 2022] DELETE /identity/v3/projects/e676a17baa754a9fa4eb6ee91149c0b2 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:41:53.080312 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a71d30-ea0b-4a0c-986e-15a680ff1f4a admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:53.083263 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-78a71d30-ea0b-4a0c-986e-15a680ff1f4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:53.084604 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4db57a6-6dc8-4ea2-8e8d-6672b84907ea service neutron] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:53.168268 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3300b979-a1f3-4756-92ad-fa5f210e95b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:53.168268 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3300b979-a1f3-4756-92ad-fa5f210e95b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:53.168268 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3300b979-a1f3-4756-92ad-fa5f210e95b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:53.168268 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3300b979-a1f3-4756-92ad-fa5f210e95b0 admin admin] PATH_INFO: `/v3/projects/8c9f8c2822bc40829857bf827b5eddb1` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:53.168875 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3300b979-a1f3-4756-92ad-fa5f210e95b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c9f8c2822bc40829857bf827b5eddb1)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:53.170136 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3300b979-a1f3-4756-92ad-fa5f210e95b0 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:53.182368 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a71d30-ea0b-4a0c-986e-15a680ff1f4a admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:53.188373 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b746e36e-69c6-4a69-996e-cc41fbeddde4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:53.189369 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b746e36e-69c6-4a69-996e-cc41fbeddde4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:53.189911 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b746e36e-69c6-4a69-996e-cc41fbeddde4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:53.190284 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b746e36e-69c6-4a69-996e-cc41fbeddde4 admin admin] PATH_INFO: `/v3/users/943205be4a664d37bc8608ce4bd9b4cf` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:53.191556 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b746e36e-69c6-4a69-996e-cc41fbeddde4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=943205be4a664d37bc8608ce4bd9b4cf)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:53.193770 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b746e36e-69c6-4a69-996e-cc41fbeddde4 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:53.469249 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-3300b979-a1f3-4756-92ad-fa5f210e95b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c9f8c2822bc40829857bf827b5eddb1', 'request_id': 'req-3300b979-a1f3-4756-92ad-fa5f210e95b0'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:53.472396 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-b746e36e-69c6-4a69-996e-cc41fbeddde4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '943205be4a664d37bc8608ce4bd9b4cf'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:53.473446 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-b746e36e-69c6-4a69-996e-cc41fbeddde4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '943205be4a664d37bc8608ce4bd9b4cf'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:53.474418 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-b746e36e-69c6-4a69-996e-cc41fbeddde4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '943205be4a664d37bc8608ce4bd9b4cf'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:53.483222 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-b746e36e-69c6-4a69-996e-cc41fbeddde4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '943205be4a664d37bc8608ce4bd9b4cf'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:53.607873 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-3300b979-a1f3-4756-92ad-fa5f210e95b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c9f8c2822bc40829857bf827b5eddb1', 'request_id': 'req-3300b979-a1f3-4756-92ad-fa5f210e95b0'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:53.618654 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-3300b979-a1f3-4756-92ad-fa5f210e95b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c9f8c2822bc40829857bf827b5eddb1', 'request_id': 'req-3300b979-a1f3-4756-92ad-fa5f210e95b0'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:53.639112 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-3300b979-a1f3-4756-92ad-fa5f210e95b0 admin admin] The token cache is being invalidate because project 8c9f8c2822bc40829857bf827b5eddb1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81723) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:41:53.639861 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-3300b979-a1f3-4756-92ad-fa5f210e95b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:53.646704 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-3300b979-a1f3-4756-92ad-fa5f210e95b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:53.648589 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1006/2004] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:41:53 2022] DELETE /identity/v3/projects/8c9f8c2822bc40829857bf827b5eddb1 => generated 0 bytes in 592 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:41:53.652684 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3300b979-a1f3-4756-92ad-fa5f210e95b0 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:53.654171 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-3300b979-a1f3-4756-92ad-fa5f210e95b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:53.719081 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3300b979-a1f3-4756-92ad-fa5f210e95b0 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:53.724081 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8898c2-5354-4721-b23b-89f36fed070d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:53.724808 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8898c2-5354-4721-b23b-89f36fed070d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:53.725252 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8898c2-5354-4721-b23b-89f36fed070d admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:53.725493 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8898c2-5354-4721-b23b-89f36fed070d admin admin] PATH_INFO: `/v3/users/446b6008e7ac4c118c47ec4f7b6dcc6d` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:53.727011 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8898c2-5354-4721-b23b-89f36fed070d admin admin] RBAC: Authorizing `identity:delete_user(user_id=446b6008e7ac4c118c47ec4f7b6dcc6d)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:53.728939 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8898c2-5354-4721-b23b-89f36fed070d admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:53.762957 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-b746e36e-69c6-4a69-996e-cc41fbeddde4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '943205be4a664d37bc8608ce4bd9b4cf'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:53.783519 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 999/2005] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:41:53 2022] DELETE /identity/v3/users/943205be4a664d37bc8608ce4bd9b4cf => generated 0 bytes in 705 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:41:54.034882 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-3e8898c2-5354-4721-b23b-89f36fed070d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '446b6008e7ac4c118c47ec4f7b6dcc6d'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:54.036614 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-3e8898c2-5354-4721-b23b-89f36fed070d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '446b6008e7ac4c118c47ec4f7b6dcc6d'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:54.037803 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-3e8898c2-5354-4721-b23b-89f36fed070d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '446b6008e7ac4c118c47ec4f7b6dcc6d'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:54.221293 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-3e8898c2-5354-4721-b23b-89f36fed070d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '446b6008e7ac4c118c47ec4f7b6dcc6d'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:54.240074 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-3e8898c2-5354-4721-b23b-89f36fed070d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '446b6008e7ac4c118c47ec4f7b6dcc6d'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:54.248709 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b746e36e-69c6-4a69-996e-cc41fbeddde4 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:54.250319 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-b746e36e-69c6-4a69-996e-cc41fbeddde4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:54.254957 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1007/2006] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:41:53 2022] DELETE /identity/v3/users/446b6008e7ac4c118c47ec4f7b6dcc6d => generated 0 bytes in 603 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:41:54.296063 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8898c2-5354-4721-b23b-89f36fed070d admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:54.300976 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-3e8898c2-5354-4721-b23b-89f36fed070d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:54.326262 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b746e36e-69c6-4a69-996e-cc41fbeddde4 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:54.406191 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8898c2-5354-4721-b23b-89f36fed070d admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:54.411516 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ef5ae1-c76b-49b6-aca9-3e21d80426ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:54.412122 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04b10b3-841f-4e02-a1b5-019c501f1065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:54.412740 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04b10b3-841f-4e02-a1b5-019c501f1065 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:54.412897 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ef5ae1-c76b-49b6-aca9-3e21d80426ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:54.413095 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04b10b3-841f-4e02-a1b5-019c501f1065 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:54.413326 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04b10b3-841f-4e02-a1b5-019c501f1065 admin admin] PATH_INFO: `/v3/projects/ad1fcd639e604251a43bd90a5a7ab2a5` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:54.413427 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ef5ae1-c76b-49b6-aca9-3e21d80426ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:54.413926 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ef5ae1-c76b-49b6-aca9-3e21d80426ec admin admin] PATH_INFO: `/v3/users/b5d6c36fb83c4955849c98cb9f96a6aa` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:54.414506 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04b10b3-841f-4e02-a1b5-019c501f1065 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad1fcd639e604251a43bd90a5a7ab2a5)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:54.415815 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ef5ae1-c76b-49b6-aca9-3e21d80426ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5d6c36fb83c4955849c98cb9f96a6aa)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:54.416455 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04b10b3-841f-4e02-a1b5-019c501f1065 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:54.419525 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ef5ae1-c76b-49b6-aca9-3e21d80426ec admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:54.621684 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-43ef5ae1-c76b-49b6-aca9-3e21d80426ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5d6c36fb83c4955849c98cb9f96a6aa'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:54.622368 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-a04b10b3-841f-4e02-a1b5-019c501f1065 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad1fcd639e604251a43bd90a5a7ab2a5', 'request_id': 'req-a04b10b3-841f-4e02-a1b5-019c501f1065'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:54.624700 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-43ef5ae1-c76b-49b6-aca9-3e21d80426ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5d6c36fb83c4955849c98cb9f96a6aa'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:54.626015 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-43ef5ae1-c76b-49b6-aca9-3e21d80426ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5d6c36fb83c4955849c98cb9f96a6aa'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:54.635651 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-43ef5ae1-c76b-49b6-aca9-3e21d80426ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5d6c36fb83c4955849c98cb9f96a6aa'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:54.723110 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-a04b10b3-841f-4e02-a1b5-019c501f1065 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad1fcd639e604251a43bd90a5a7ab2a5', 'request_id': 'req-a04b10b3-841f-4e02-a1b5-019c501f1065'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:54.731858 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-a04b10b3-841f-4e02-a1b5-019c501f1065 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad1fcd639e604251a43bd90a5a7ab2a5', 'request_id': 'req-a04b10b3-841f-4e02-a1b5-019c501f1065'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:54.754064 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-a04b10b3-841f-4e02-a1b5-019c501f1065 admin admin] The token cache is being invalidate because project ad1fcd639e604251a43bd90a5a7ab2a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81723) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:41:54.754474 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-a04b10b3-841f-4e02-a1b5-019c501f1065 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:54.755485 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-a04b10b3-841f-4e02-a1b5-019c501f1065 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:54.757076 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1008/2007] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:41:54 2022] DELETE /identity/v3/projects/ad1fcd639e604251a43bd90a5a7ab2a5 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:41:54.762009 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04b10b3-841f-4e02-a1b5-019c501f1065 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:54.763599 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-a04b10b3-841f-4e02-a1b5-019c501f1065 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:54.824356 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-43ef5ae1-c76b-49b6-aca9-3e21d80426ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5d6c36fb83c4955849c98cb9f96a6aa'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:54.846841 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1000/2008] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:41:54 2022] DELETE /identity/v3/users/b5d6c36fb83c4955849c98cb9f96a6aa => generated 0 bytes in 599 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:41:54.847820 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04b10b3-841f-4e02-a1b5-019c501f1065 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:54.851152 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ef5ae1-c76b-49b6-aca9-3e21d80426ec admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:54.852481 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-43ef5ae1-c76b-49b6-aca9-3e21d80426ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:54.923101 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ef5ae1-c76b-49b6-aca9-3e21d80426ec admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:54.927499 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c50edb-3576-4acf-8aef-67bf17ae9ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:54.928254 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c50edb-3576-4acf-8aef-67bf17ae9ad2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:54.931865 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab529032-adc8-45dc-aec0-56664398b00f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:54.931984 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c50edb-3576-4acf-8aef-67bf17ae9ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:54.931984 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c50edb-3576-4acf-8aef-67bf17ae9ad2 admin admin] PATH_INFO: `/v3/users/091e7cea39ff42c9b730907d12b210cc` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:54.932171 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab529032-adc8-45dc-aec0-56664398b00f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:54.932171 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab529032-adc8-45dc-aec0-56664398b00f admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:54.932171 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab529032-adc8-45dc-aec0-56664398b00f admin admin] PATH_INFO: `/v3/projects/4bf396234e3540198914d2bc13697ffa` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:54.932603 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c50edb-3576-4acf-8aef-67bf17ae9ad2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=091e7cea39ff42c9b730907d12b210cc)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:54.933296 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab529032-adc8-45dc-aec0-56664398b00f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4bf396234e3540198914d2bc13697ffa)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:54.933446 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c50edb-3576-4acf-8aef-67bf17ae9ad2 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:54.933570 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab529032-adc8-45dc-aec0-56664398b00f admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:55.197200 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:41:55.270089 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-ab529032-adc8-45dc-aec0-56664398b00f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bf396234e3540198914d2bc13697ffa', 'request_id': 'req-ab529032-adc8-45dc-aec0-56664398b00f'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:55.270509 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-a2c50edb-3576-4acf-8aef-67bf17ae9ad2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '091e7cea39ff42c9b730907d12b210cc'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:55.271199 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-a2c50edb-3576-4acf-8aef-67bf17ae9ad2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '091e7cea39ff42c9b730907d12b210cc'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:55.272045 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-a2c50edb-3576-4acf-8aef-67bf17ae9ad2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '091e7cea39ff42c9b730907d12b210cc'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:55.275566 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-ab529032-adc8-45dc-aec0-56664398b00f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bf396234e3540198914d2bc13697ffa', 'request_id': 'req-ab529032-adc8-45dc-aec0-56664398b00f'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:55.383614 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-a2c50edb-3576-4acf-8aef-67bf17ae9ad2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '091e7cea39ff42c9b730907d12b210cc'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:55.397399 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-a2c50edb-3576-4acf-8aef-67bf17ae9ad2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '091e7cea39ff42c9b730907d12b210cc'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:55.409552 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1009/2009] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:41:54 2022] DELETE /identity/v3/users/091e7cea39ff42c9b730907d12b210cc => generated 0 bytes in 649 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:41:55.410857 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024f0a2f-48ee-4ba9-8f14-b95a6d9d5a3a None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:55.411146 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024f0a2f-48ee-4ba9-8f14-b95a6d9d5a3a None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:55.411413 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024f0a2f-48ee-4ba9-8f14-b95a6d9d5a3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:55.434575 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-024f0a2f-48ee-4ba9-8f14-b95a6d9d5a3a None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:41:55.436723 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-024f0a2f-48ee-4ba9-8f14-b95a6d9d5a3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:55.494012 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-ab529032-adc8-45dc-aec0-56664398b00f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bf396234e3540198914d2bc13697ffa', 'request_id': 'req-ab529032-adc8-45dc-aec0-56664398b00f'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:55.507757 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-ab529032-adc8-45dc-aec0-56664398b00f admin admin] The token cache is being invalidate because project 4bf396234e3540198914d2bc13697ffa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:41:55.508059 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-ab529032-adc8-45dc-aec0-56664398b00f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:55.508750 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-ab529032-adc8-45dc-aec0-56664398b00f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:55.512014 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1001/2010] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:41:54 2022] DELETE /identity/v3/projects/4bf396234e3540198914d2bc13697ffa => generated 0 bytes in 663 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:41:55.520273 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab529032-adc8-45dc-aec0-56664398b00f admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:55.521835 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-ab529032-adc8-45dc-aec0-56664398b00f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:55.523843 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1010/2011] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:41:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:41:55.535515 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024f0a2f-48ee-4ba9-8f14-b95a6d9d5a3a None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:55.537049 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-024f0a2f-48ee-4ba9-8f14-b95a6d9d5a3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:55.591036 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab529032-adc8-45dc-aec0-56664398b00f admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:55.597200 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fed0a73-16a0-4039-ad12-55de597b0685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:55.602422 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fed0a73-16a0-4039-ad12-55de597b0685 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:55.602422 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fed0a73-16a0-4039-ad12-55de597b0685 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:55.602422 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fed0a73-16a0-4039-ad12-55de597b0685 admin admin] PATH_INFO: `/v3/users/eacae54b966944e28c112fe0efbdbc65` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:55.602422 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fed0a73-16a0-4039-ad12-55de597b0685 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eacae54b966944e28c112fe0efbdbc65)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:55.602422 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fed0a73-16a0-4039-ad12-55de597b0685 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:55.610957 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024f0a2f-48ee-4ba9-8f14-b95a6d9d5a3a None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:55.614676 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002fffc3-9b9f-42e6-b696-b726d9417d80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:55.615284 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002fffc3-9b9f-42e6-b696-b726d9417d80 service neutron] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:55.615566 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002fffc3-9b9f-42e6-b696-b726d9417d80 service neutron] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:55.615837 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002fffc3-9b9f-42e6-b696-b726d9417d80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:55.626645 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002fffc3-9b9f-42e6-b696-b726d9417d80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:55.628191 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002fffc3-9b9f-42e6-b696-b726d9417d80 service neutron] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:55.638400 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1011/2012] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:41:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:41:55.643034 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:41:55.879050 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-6fed0a73-16a0-4039-ad12-55de597b0685 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eacae54b966944e28c112fe0efbdbc65'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:55.881051 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-6fed0a73-16a0-4039-ad12-55de597b0685 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eacae54b966944e28c112fe0efbdbc65'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:55.881740 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-6fed0a73-16a0-4039-ad12-55de597b0685 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eacae54b966944e28c112fe0efbdbc65'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:55.893271 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-6fed0a73-16a0-4039-ad12-55de597b0685 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eacae54b966944e28c112fe0efbdbc65'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:55.985325 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002fffc3-9b9f-42e6-b696-b726d9417d80 service neutron] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:55.987927 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-002fffc3-9b9f-42e6-b696-b726d9417d80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:56.007472 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002fffc3-9b9f-42e6-b696-b726d9417d80 service neutron] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:56.008953 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=80,keystone:INSERT=3,keystone:DELETE=17 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:41:56.011556 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-6fed0a73-16a0-4039-ad12-55de597b0685 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eacae54b966944e28c112fe0efbdbc65'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:56.016160 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff4775e-bd00-4540-81da-4c7b865cb2cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:56.016160 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff4775e-bd00-4540-81da-4c7b865cb2cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:56.016160 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff4775e-bd00-4540-81da-4c7b865cb2cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:56.016160 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff4775e-bd00-4540-81da-4c7b865cb2cf admin admin] PATH_INFO: `/v3/projects/92c23c7df11e4f259b7a4d9b9facccab` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:56.016691 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff4775e-bd00-4540-81da-4c7b865cb2cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=92c23c7df11e4f259b7a4d9b9facccab)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:56.017614 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff4775e-bd00-4540-81da-4c7b865cb2cf admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:56.049786 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1002/2013] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:41:55 2022] DELETE /identity/v3/users/eacae54b966944e28c112fe0efbdbc65 => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:41:56.164696 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-bff4775e-bd00-4540-81da-4c7b865cb2cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92c23c7df11e4f259b7a4d9b9facccab', 'request_id': 'req-bff4775e-bd00-4540-81da-4c7b865cb2cf'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:56.231275 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-bff4775e-bd00-4540-81da-4c7b865cb2cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92c23c7df11e4f259b7a4d9b9facccab', 'request_id': 'req-bff4775e-bd00-4540-81da-4c7b865cb2cf'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:56.240299 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fed0a73-16a0-4039-ad12-55de597b0685 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:56.243002 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-6fed0a73-16a0-4039-ad12-55de597b0685 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:56.257426 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-bff4775e-bd00-4540-81da-4c7b865cb2cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92c23c7df11e4f259b7a4d9b9facccab', 'request_id': 'req-bff4775e-bd00-4540-81da-4c7b865cb2cf'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:56.284271 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-bff4775e-bd00-4540-81da-4c7b865cb2cf admin admin] The token cache is being invalidate because project 92c23c7df11e4f259b7a4d9b9facccab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81723) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:41:56.284271 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-bff4775e-bd00-4540-81da-4c7b865cb2cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:56.284271 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-bff4775e-bd00-4540-81da-4c7b865cb2cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:56.289342 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1012/2014] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:41:55 2022] DELETE /identity/v3/projects/92c23c7df11e4f259b7a4d9b9facccab => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:41:56.290834 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c6015c-bad8-447d-a2bb-797f63bd5f68 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:56.291402 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c6015c-bad8-447d-a2bb-797f63bd5f68 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:56.291690 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c6015c-bad8-447d-a2bb-797f63bd5f68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:56.333984 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-35c6015c-bad8-447d-a2bb-797f63bd5f68 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:41:56.335491 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-35c6015c-bad8-447d-a2bb-797f63bd5f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:56.352720 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fed0a73-16a0-4039-ad12-55de597b0685 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:56.423629 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1013/2015] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:41:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:41:56.431762 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fff1754-68f4-4f5d-84e0-e93a39ad8f9a None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:56.432839 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fff1754-68f4-4f5d-84e0-e93a39ad8f9a None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:56.432839 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fff1754-68f4-4f5d-84e0-e93a39ad8f9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:56.452073 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341f7a22-2aa6-4b56-ad52-93a7b1180472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:56.452830 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341f7a22-2aa6-4b56-ad52-93a7b1180472 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:56.453120 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341f7a22-2aa6-4b56-ad52-93a7b1180472 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:56.453431 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341f7a22-2aa6-4b56-ad52-93a7b1180472 admin admin] PATH_INFO: `/v3/projects/9f9adde8befc4c25a2fde3c26622ce83` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:56.454598 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-0fff1754-68f4-4f5d-84e0-e93a39ad8f9a None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:41:56.455482 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341f7a22-2aa6-4b56-ad52-93a7b1180472 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f9adde8befc4c25a2fde3c26622ce83)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:56.456417 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-0fff1754-68f4-4f5d-84e0-e93a39ad8f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:56.457989 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341f7a22-2aa6-4b56-ad52-93a7b1180472 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:56.468979 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1014/2016] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:41:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:41:56.473277 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fff1754-68f4-4f5d-84e0-e93a39ad8f9a None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:56.483510 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fff1754-68f4-4f5d-84e0-e93a39ad8f9a None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:56.488257 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1983e0ee-a306-454e-92b6-ecf6145c2766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:56.488871 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1983e0ee-a306-454e-92b6-ecf6145c2766 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:56.489161 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1983e0ee-a306-454e-92b6-ecf6145c2766 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:56.489467 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1983e0ee-a306-454e-92b6-ecf6145c2766 admin admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:56.490116 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1983e0ee-a306-454e-92b6-ecf6145c2766 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:41:56.490420 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1983e0ee-a306-454e-92b6-ecf6145c2766 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:56.492065 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1983e0ee-a306-454e-92b6-ecf6145c2766 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:56.504991 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1015/2017] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 24 06:41:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:41:56.509474 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1983e0ee-a306-454e-92b6-ecf6145c2766 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:56.519562 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1983e0ee-a306-454e-92b6-ecf6145c2766 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:56.524638 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8cc155-5106-43d9-b3bb-671d268a8446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:56.525251 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8cc155-5106-43d9-b3bb-671d268a8446 admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:56.525557 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8cc155-5106-43d9-b3bb-671d268a8446 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:56.525832 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8cc155-5106-43d9-b3bb-671d268a8446 admin admin] PATH_INFO: `/v3/projects` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:56.526629 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8cc155-5106-43d9-b3bb-671d268a8446 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:56.528574 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8cc155-5106-43d9-b3bb-671d268a8446 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:56.642480 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-341f7a22-2aa6-4b56-ad52-93a7b1180472 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f9adde8befc4c25a2fde3c26622ce83', 'request_id': 'req-341f7a22-2aa6-4b56-ad52-93a7b1180472'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:56.648741 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-341f7a22-2aa6-4b56-ad52-93a7b1180472 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f9adde8befc4c25a2fde3c26622ce83', 'request_id': 'req-341f7a22-2aa6-4b56-ad52-93a7b1180472'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:56.654363 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1016/2018] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:41:56 2022] POST /identity/v3/projects => generated 373 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:41:56.657920 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8cc155-5106-43d9-b3bb-671d268a8446 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:56.657920 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-1b8cc155-5106-43d9-b3bb-671d268a8446 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:56.727219 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8cc155-5106-43d9-b3bb-671d268a8446 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:56.733275 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf554f5c-fcce-40af-82b2-86da27b167a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:56.733919 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf554f5c-fcce-40af-82b2-86da27b167a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:56.734224 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf554f5c-fcce-40af-82b2-86da27b167a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:56.734511 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf554f5c-fcce-40af-82b2-86da27b167a6 admin admin] PATH_INFO: `/v3/projects/fbe7372b867e475bb787f41afd2e3b49` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:56.735664 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf554f5c-fcce-40af-82b2-86da27b167a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbe7372b867e475bb787f41afd2e3b49)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:56.737730 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf554f5c-fcce-40af-82b2-86da27b167a6 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:56.762389 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-341f7a22-2aa6-4b56-ad52-93a7b1180472 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f9adde8befc4c25a2fde3c26622ce83', 'request_id': 'req-341f7a22-2aa6-4b56-ad52-93a7b1180472'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:56.779367 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-341f7a22-2aa6-4b56-ad52-93a7b1180472 admin admin] The token cache is being invalidate because project 9f9adde8befc4c25a2fde3c26622ce83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:41:56.779367 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-341f7a22-2aa6-4b56-ad52-93a7b1180472 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:56.779367 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-341f7a22-2aa6-4b56-ad52-93a7b1180472 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:56.779367 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1003/2019] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:41:56 2022] DELETE /identity/v3/projects/9f9adde8befc4c25a2fde3c26622ce83 => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:41:56.779947 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341f7a22-2aa6-4b56-ad52-93a7b1180472 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:56.781342 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-341f7a22-2aa6-4b56-ad52-93a7b1180472 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:56.858516 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341f7a22-2aa6-4b56-ad52-93a7b1180472 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:56.863335 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a96a0c-96c7-4975-baa0-afc598cbf7ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:56.864106 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a96a0c-96c7-4975-baa0-afc598cbf7ff admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:56.864398 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a96a0c-96c7-4975-baa0-afc598cbf7ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:56.864672 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a96a0c-96c7-4975-baa0-afc598cbf7ff admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:56.865333 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a96a0c-96c7-4975-baa0-afc598cbf7ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:56.867269 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a96a0c-96c7-4975-baa0-afc598cbf7ff admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:56.903939 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-cf554f5c-fcce-40af-82b2-86da27b167a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbe7372b867e475bb787f41afd2e3b49', 'request_id': 'req-cf554f5c-fcce-40af-82b2-86da27b167a6'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:57.069437 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-cf554f5c-fcce-40af-82b2-86da27b167a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbe7372b867e475bb787f41afd2e3b49', 'request_id': 'req-cf554f5c-fcce-40af-82b2-86da27b167a6'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:57.077098 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-cf554f5c-fcce-40af-82b2-86da27b167a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbe7372b867e475bb787f41afd2e3b49', 'request_id': 'req-cf554f5c-fcce-40af-82b2-86da27b167a6'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:57.092150 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-cf554f5c-fcce-40af-82b2-86da27b167a6 admin admin] The token cache is being invalidate because project fbe7372b867e475bb787f41afd2e3b49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81723) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:41:57.092443 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-cf554f5c-fcce-40af-82b2-86da27b167a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:57.093130 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-cf554f5c-fcce-40af-82b2-86da27b167a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:57.094721 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1017/2020] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:41:56 2022] DELETE /identity/v3/projects/fbe7372b867e475bb787f41afd2e3b49 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:41:57.095569 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf554f5c-fcce-40af-82b2-86da27b167a6 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:57.096886 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-cf554f5c-fcce-40af-82b2-86da27b167a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:57.170324 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf554f5c-fcce-40af-82b2-86da27b167a6 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:57.175620 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1979424f-c1ac-40c7-837c-4c5b74a54dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:57.176295 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1979424f-c1ac-40c7-837c-4c5b74a54dce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:57.176619 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1979424f-c1ac-40c7-837c-4c5b74a54dce admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:57.176918 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1979424f-c1ac-40c7-837c-4c5b74a54dce admin admin] PATH_INFO: `/v3/users/9120eb33e6904c5c845a3c59448bc551` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:57.178127 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1979424f-c1ac-40c7-837c-4c5b74a54dce admin admin] RBAC: Authorizing `identity:delete_user(user_id=9120eb33e6904c5c845a3c59448bc551)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:57.180153 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1979424f-c1ac-40c7-837c-4c5b74a54dce admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:57.233301 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1004/2021] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:41:56 2022] POST /identity/v3/users => generated 287 bytes in 454 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:41:57.235190 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbc65c9-0790-4e48-afaf-8d66cf60bc6c None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:57.235586 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbc65c9-0790-4e48-afaf-8d66cf60bc6c None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:57.235880 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbc65c9-0790-4e48-afaf-8d66cf60bc6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:57.256813 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-acbc65c9-0790-4e48-afaf-8d66cf60bc6c None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:41:57.258135 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-acbc65c9-0790-4e48-afaf-8d66cf60bc6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:57.267142 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1005/2022] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:41:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:41:57.268351 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbc65c9-0790-4e48-afaf-8d66cf60bc6c None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:57.274055 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-acbc65c9-0790-4e48-afaf-8d66cf60bc6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:57.282967 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbc65c9-0790-4e48-afaf-8d66cf60bc6c None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:57.287939 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b97b98e-0e1c-492a-9b78-82871198dce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:57.288604 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b97b98e-0e1c-492a-9b78-82871198dce3 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:57.288887 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b97b98e-0e1c-492a-9b78-82871198dce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:57.289223 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b97b98e-0e1c-492a-9b78-82871198dce3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:57.289894 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b97b98e-0e1c-492a-9b78-82871198dce3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:57.291547 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b97b98e-0e1c-492a-9b78-82871198dce3 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:57.302740 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1006/2023] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:41:57 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:41:57.303987 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666b736d-e340-48f4-854e-23dd875e7bec None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:57.304277 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666b736d-e340-48f4-854e-23dd875e7bec None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:57.304582 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666b736d-e340-48f4-854e-23dd875e7bec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:57.327505 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-666b736d-e340-48f4-854e-23dd875e7bec None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:41:57.329238 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-666b736d-e340-48f4-854e-23dd875e7bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:57.341802 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1007/2024] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:41:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:41:57.342661 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666b736d-e340-48f4-854e-23dd875e7bec None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:57.353306 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666b736d-e340-48f4-854e-23dd875e7bec None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:57.358888 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e28cf1-94fb-4e49-9a51-11bbe3072a7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:57.359504 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e28cf1-94fb-4e49-9a51-11bbe3072a7f admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:57.359823 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e28cf1-94fb-4e49-9a51-11bbe3072a7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:57.360099 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e28cf1-94fb-4e49-9a51-11bbe3072a7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:57.360747 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e28cf1-94fb-4e49-9a51-11bbe3072a7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:57.362364 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e28cf1-94fb-4e49-9a51-11bbe3072a7f admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:57.373453 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1008/2025] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:41:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:41:57.374291 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e28cf1-94fb-4e49-9a51-11bbe3072a7f admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:57.384104 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e28cf1-94fb-4e49-9a51-11bbe3072a7f admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:57.389970 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6277658-4696-4307-8c75-dee42f30b474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:57.390692 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6277658-4696-4307-8c75-dee42f30b474 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:57.390979 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6277658-4696-4307-8c75-dee42f30b474 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:57.391252 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6277658-4696-4307-8c75-dee42f30b474 admin admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:57.391906 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6277658-4696-4307-8c75-dee42f30b474 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:41:57.392208 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6277658-4696-4307-8c75-dee42f30b474 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:57.393846 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6277658-4696-4307-8c75-dee42f30b474 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:57.407524 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1009/2026] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 24 06:41:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:41:57.408373 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6277658-4696-4307-8c75-dee42f30b474 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:57.419523 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6277658-4696-4307-8c75-dee42f30b474 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:57.424190 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cb8066-6afe-4628-bfdb-5a43ca6a97e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:57.424836 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cb8066-6afe-4628-bfdb-5a43ca6a97e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:57.425117 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cb8066-6afe-4628-bfdb-5a43ca6a97e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:57.425425 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cb8066-6afe-4628-bfdb-5a43ca6a97e3 admin admin] PATH_INFO: `/v3/projects/7bf6c50b3a7a446ba5e001bee89b7595/users/bd7a4531f4874786a8aa16581c4feea5/roles/e1f9b167c51c4a388cbc9594db8a465d` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:57.442809 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cb8066-6afe-4628-bfdb-5a43ca6a97e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bf6c50b3a7a446ba5e001bee89b7595, user_id=bd7a4531f4874786a8aa16581c4feea5, role_id=e1f9b167c51c4a388cbc9594db8a465d)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:57.446195 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cb8066-6afe-4628-bfdb-5a43ca6a97e3 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:57.548141 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1010/2027] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:41:57 2022] PUT /identity/v3/projects/7bf6c50b3a7a446ba5e001bee89b7595/users/bd7a4531f4874786a8aa16581c4feea5/roles/e1f9b167c51c4a388cbc9594db8a465d => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:41:57.549329 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cb8066-6afe-4628-bfdb-5a43ca6a97e3 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:57.555417 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-1979424f-c1ac-40c7-837c-4c5b74a54dce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9120eb33e6904c5c845a3c59448bc551'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:57.556075 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-1979424f-c1ac-40c7-837c-4c5b74a54dce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9120eb33e6904c5c845a3c59448bc551'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:57.556719 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-1979424f-c1ac-40c7-837c-4c5b74a54dce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9120eb33e6904c5c845a3c59448bc551'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:57.559034 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-85cb8066-6afe-4628-bfdb-5a43ca6a97e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:57.629678 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cb8066-6afe-4628-bfdb-5a43ca6a97e3 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:57.634970 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c254c6-a1b4-4940-acd0-b400057bd8ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:57.635643 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c254c6-a1b4-4940-acd0-b400057bd8ed admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:57.635933 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c254c6-a1b4-4940-acd0-b400057bd8ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:57.636213 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c254c6-a1b4-4940-acd0-b400057bd8ed admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:57.636904 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c254c6-a1b4-4940-acd0-b400057bd8ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:57.639674 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c254c6-a1b4-4940-acd0-b400057bd8ed admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:57.640935 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-1979424f-c1ac-40c7-837c-4c5b74a54dce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9120eb33e6904c5c845a3c59448bc551'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:57.656255 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-1979424f-c1ac-40c7-837c-4c5b74a54dce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9120eb33e6904c5c845a3c59448bc551'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:57.670492 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1018/2028] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:41:57 2022] DELETE /identity/v3/users/9120eb33e6904c5c845a3c59448bc551 => generated 0 bytes in 575 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:41:57.671747 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c015e54-29da-4763-8ed0-66a3bc4dcb66 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:57.672034 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c015e54-29da-4763-8ed0-66a3bc4dcb66 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:57.672318 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c015e54-29da-4763-8ed0-66a3bc4dcb66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:57.702122 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-0c015e54-29da-4763-8ed0-66a3bc4dcb66 None None] MFA Rules not processed for user `bd7a4531f4874786a8aa16581c4feea5`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:41:57.706779 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-0c015e54-29da-4763-8ed0-66a3bc4dcb66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:57.773225 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1011/2029] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:41:57 2022] POST /identity/v3/projects => generated 401 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:41:57.773455 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1019/2030] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:41:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:41:57.775170 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c254c6-a1b4-4940-acd0-b400057bd8ed admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:57.785454 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c015e54-29da-4763-8ed0-66a3bc4dcb66 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:57.787017 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-0c015e54-29da-4763-8ed0-66a3bc4dcb66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:57.856338 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c015e54-29da-4763-8ed0-66a3bc4dcb66 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:57.864644 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c254c6-a1b4-4940-acd0-b400057bd8ed admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:57.874421 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a06cdb-e29e-43e6-9bd7-f4ae18a7f6db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:57.874421 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a06cdb-e29e-43e6-9bd7-f4ae18a7f6db admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:57.874421 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a06cdb-e29e-43e6-9bd7-f4ae18a7f6db admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:57.874421 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a06cdb-e29e-43e6-9bd7-f4ae18a7f6db admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:57.874889 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a06cdb-e29e-43e6-9bd7-f4ae18a7f6db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:57.877426 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a06cdb-e29e-43e6-9bd7-f4ae18a7f6db admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:57.924439 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68521122-2d36-4832-97f1-fc7c7e1f9708 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36639dce4f97412f89d71367f650b901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:57.925532 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68521122-2d36-4832-97f1-fc7c7e1f9708 service cinder] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:57.925814 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68521122-2d36-4832-97f1-fc7c7e1f9708 service cinder] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:57.926559 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68521122-2d36-4832-97f1-fc7c7e1f9708 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:57.987485 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68521122-2d36-4832-97f1-fc7c7e1f9708 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:57.989120 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68521122-2d36-4832-97f1-fc7c7e1f9708 service cinder] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:57.999728 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1020/2031] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:41:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:41:58.000523 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68521122-2d36-4832-97f1-fc7c7e1f9708 service cinder] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:58.005165 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-68521122-2d36-4832-97f1-fc7c7e1f9708 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:58.019694 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68521122-2d36-4832-97f1-fc7c7e1f9708 service cinder] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:58.025107 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624f510f-014f-4810-98da-26c625787966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:58.025807 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624f510f-014f-4810-98da-26c625787966 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:58.026101 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624f510f-014f-4810-98da-26c625787966 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:58.026407 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624f510f-014f-4810-98da-26c625787966 admin admin] PATH_INFO: `/v3/users/dba7c5b5bb0848caaa5e139fd711c617` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:58.028689 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624f510f-014f-4810-98da-26c625787966 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dba7c5b5bb0848caaa5e139fd711c617)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:58.030776 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624f510f-014f-4810-98da-26c625787966 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:58.043971 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1012/2032] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:41:57 2022] POST /identity/v3/users => generated 301 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:41:58.046024 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a06cdb-e29e-43e6-9bd7-f4ae18a7f6db admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:58.046514 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-b0a06cdb-e29e-43e6-9bd7-f4ae18a7f6db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:58.059617 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a06cdb-e29e-43e6-9bd7-f4ae18a7f6db admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:58.064708 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b60cc0-027c-4230-9c80-ac46cc36464a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:58.065400 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b60cc0-027c-4230-9c80-ac46cc36464a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:58.065677 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b60cc0-027c-4230-9c80-ac46cc36464a admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:58.065952 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b60cc0-027c-4230-9c80-ac46cc36464a admin admin] PATH_INFO: `/v3/users/bd7a4531f4874786a8aa16581c4feea5` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:58.067158 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b60cc0-027c-4230-9c80-ac46cc36464a admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd7a4531f4874786a8aa16581c4feea5)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:58.069142 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b60cc0-027c-4230-9c80-ac46cc36464a admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:58.474262 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-624f510f-014f-4810-98da-26c625787966 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dba7c5b5bb0848caaa5e139fd711c617'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:58.474921 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-624f510f-014f-4810-98da-26c625787966 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dba7c5b5bb0848caaa5e139fd711c617'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:58.475652 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-624f510f-014f-4810-98da-26c625787966 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dba7c5b5bb0848caaa5e139fd711c617'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:58.475780 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-c1b60cc0-027c-4230-9c80-ac46cc36464a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd7a4531f4874786a8aa16581c4feea5'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:58.476655 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-c1b60cc0-027c-4230-9c80-ac46cc36464a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd7a4531f4874786a8aa16581c4feea5'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:58.477465 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-c1b60cc0-027c-4230-9c80-ac46cc36464a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd7a4531f4874786a8aa16581c4feea5'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:58.486230 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-c1b60cc0-027c-4230-9c80-ac46cc36464a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd7a4531f4874786a8aa16581c4feea5'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:58.584087 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:41:58.617623 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-624f510f-014f-4810-98da-26c625787966 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dba7c5b5bb0848caaa5e139fd711c617'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:58.631482 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-624f510f-014f-4810-98da-26c625787966 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dba7c5b5bb0848caaa5e139fd711c617'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:58.645459 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1021/2033] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:41:57 2022] DELETE /identity/v3/users/dba7c5b5bb0848caaa5e139fd711c617 => generated 0 bytes in 645 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:41:58.646296 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624f510f-014f-4810-98da-26c625787966 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:58.648345 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-624f510f-014f-4810-98da-26c625787966 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:58.720867 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-c1b60cc0-027c-4230-9c80-ac46cc36464a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd7a4531f4874786a8aa16581c4feea5'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:58.731905 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624f510f-014f-4810-98da-26c625787966 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:58.739854 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe71ce0-878e-46c7-94f0-a508b8222e75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:58.739854 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe71ce0-878e-46c7-94f0-a508b8222e75 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:58.739854 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe71ce0-878e-46c7-94f0-a508b8222e75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:58.739854 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe71ce0-878e-46c7-94f0-a508b8222e75 admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:58.740500 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe71ce0-878e-46c7-94f0-a508b8222e75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:58.740852 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe71ce0-878e-46c7-94f0-a508b8222e75 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:58.748103 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1013/2034] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:41:58 2022] DELETE /identity/v3/users/bd7a4531f4874786a8aa16581c4feea5 => generated 0 bytes in 699 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:41:58.748103 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b60cc0-027c-4230-9c80-ac46cc36464a admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:58.748103 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-c1b60cc0-027c-4230-9c80-ac46cc36464a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:58.752565 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1022/2035] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:41:58 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:41:58.756562 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce497830-a979-4c3b-a96f-2705a126cdf0 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:58.756851 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce497830-a979-4c3b-a96f-2705a126cdf0 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:58.757137 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce497830-a979-4c3b-a96f-2705a126cdf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:58.779066 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-ce497830-a979-4c3b-a96f-2705a126cdf0 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:41:58.797655 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-ce497830-a979-4c3b-a96f-2705a126cdf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:58.823578 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:41:58.845095 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b60cc0-027c-4230-9c80-ac46cc36464a admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:58.850751 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1296503-68a2-4319-a785-0eeec0f85b52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:58.851562 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1296503-68a2-4319-a785-0eeec0f85b52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:58.852028 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1296503-68a2-4319-a785-0eeec0f85b52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:58.852353 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1296503-68a2-4319-a785-0eeec0f85b52 admin admin] PATH_INFO: `/v3/projects/da369e0dd1eb43cab32ea0cd064a28d4` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:58.853634 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1296503-68a2-4319-a785-0eeec0f85b52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da369e0dd1eb43cab32ea0cd064a28d4)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:58.856362 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1296503-68a2-4319-a785-0eeec0f85b52 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:58.858259 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1023/2036] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:41:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:41:58.866042 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1afd0f0-557a-428a-b2c4-0ed3116f4186 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:58.866042 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1afd0f0-557a-428a-b2c4-0ed3116f4186 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:58.866042 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1afd0f0-557a-428a-b2c4-0ed3116f4186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:58.888352 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-a1afd0f0-557a-428a-b2c4-0ed3116f4186 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:41:58.888352 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-a1afd0f0-557a-428a-b2c4-0ed3116f4186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:58.895922 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1024/2037] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:41:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:41:58.896789 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1afd0f0-557a-428a-b2c4-0ed3116f4186 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:58.900010 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-a1afd0f0-557a-428a-b2c4-0ed3116f4186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:58.910718 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1afd0f0-557a-428a-b2c4-0ed3116f4186 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:58.917808 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21eb148-a5a7-40b4-b39e-c569014cfc2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:58.918588 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21eb148-a5a7-40b4-b39e-c569014cfc2d admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:58.918870 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21eb148-a5a7-40b4-b39e-c569014cfc2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:58.919151 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21eb148-a5a7-40b4-b39e-c569014cfc2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:58.919807 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21eb148-a5a7-40b4-b39e-c569014cfc2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:58.921530 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21eb148-a5a7-40b4-b39e-c569014cfc2d admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:58.938210 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1025/2038] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:41:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:41:58.938210 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21eb148-a5a7-40b4-b39e-c569014cfc2d admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:58.938210 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-d21eb148-a5a7-40b4-b39e-c569014cfc2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:58.996369 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21eb148-a5a7-40b4-b39e-c569014cfc2d admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:59.015075 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efccff40-17f3-4c3f-b36a-a320a27f6ce4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:59.015075 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efccff40-17f3-4c3f-b36a-a320a27f6ce4 service neutron] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:59.015075 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efccff40-17f3-4c3f-b36a-a320a27f6ce4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:59.015075 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efccff40-17f3-4c3f-b36a-a320a27f6ce4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:59.015749 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efccff40-17f3-4c3f-b36a-a320a27f6ce4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:59.015749 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efccff40-17f3-4c3f-b36a-a320a27f6ce4 service neutron] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:59.024316 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1026/2039] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:41:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:41:59.025541 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efccff40-17f3-4c3f-b36a-a320a27f6ce4 service neutron] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:59.034637 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efccff40-17f3-4c3f-b36a-a320a27f6ce4 service neutron] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:59.038520 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb3e307-2d8d-472a-88f9-a7f0c230399f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:59.039123 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb3e307-2d8d-472a-88f9-a7f0c230399f service neutron] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:59.039435 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb3e307-2d8d-472a-88f9-a7f0c230399f service neutron] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:59.039721 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb3e307-2d8d-472a-88f9-a7f0c230399f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:59.051712 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb3e307-2d8d-472a-88f9-a7f0c230399f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:59.053215 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb3e307-2d8d-472a-88f9-a7f0c230399f service neutron] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:59.064842 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1027/2040] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:41:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:41:59.066372 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb3e307-2d8d-472a-88f9-a7f0c230399f service neutron] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:59.082617 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb3e307-2d8d-472a-88f9-a7f0c230399f service neutron] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:59.096868 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-b1296503-68a2-4319-a785-0eeec0f85b52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da369e0dd1eb43cab32ea0cd064a28d4', 'request_id': 'req-b1296503-68a2-4319-a785-0eeec0f85b52'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:59.108985 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-b1296503-68a2-4319-a785-0eeec0f85b52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da369e0dd1eb43cab32ea0cd064a28d4', 'request_id': 'req-b1296503-68a2-4319-a785-0eeec0f85b52'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:59.154066 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d35e0a-f992-4704-91db-5d4016265757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:59.155024 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d35e0a-f992-4704-91db-5d4016265757 admin admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:59.155144 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d35e0a-f992-4704-91db-5d4016265757 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:59.155329 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d35e0a-f992-4704-91db-5d4016265757 admin admin] PATH_INFO: `/v3/projects/e11fa7a71be1441d9045380ae0e38fc0/users/c36a5eaf154c463b965e5cd85ce2ab8c/roles/e1f9b167c51c4a388cbc9594db8a465d` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:59.172266 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d35e0a-f992-4704-91db-5d4016265757 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e11fa7a71be1441d9045380ae0e38fc0, user_id=c36a5eaf154c463b965e5cd85ce2ab8c, role_id=e1f9b167c51c4a388cbc9594db8a465d)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:59.175234 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d35e0a-f992-4704-91db-5d4016265757 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:59.309608 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-b1296503-68a2-4319-a785-0eeec0f85b52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da369e0dd1eb43cab32ea0cd064a28d4', 'request_id': 'req-b1296503-68a2-4319-a785-0eeec0f85b52'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:59.329412 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-b1296503-68a2-4319-a785-0eeec0f85b52 admin admin] The token cache is being invalidate because project da369e0dd1eb43cab32ea0cd064a28d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:41:59.330244 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-b1296503-68a2-4319-a785-0eeec0f85b52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:59.331309 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-b1296503-68a2-4319-a785-0eeec0f85b52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:41:59.334025 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1014/2041] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:41:58 2022] DELETE /identity/v3/projects/da369e0dd1eb43cab32ea0cd064a28d4 => generated 0 bytes in 589 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:41:59.342376 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dc6277-a454-48d3-8c92-e1e66db6c86b None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:59.342877 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dc6277-a454-48d3-8c92-e1e66db6c86b None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:59.343435 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dc6277-a454-48d3-8c92-e1e66db6c86b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:59.352971 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1028/2042] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:41:59 2022] PUT /identity/v3/projects/e11fa7a71be1441d9045380ae0e38fc0/users/c36a5eaf154c463b965e5cd85ce2ab8c/roles/e1f9b167c51c4a388cbc9594db8a465d => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:41:59.366796 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff92702-322b-4552-bf78-24e5d0ea6652 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:59.367320 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff92702-322b-4552-bf78-24e5d0ea6652 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:59.370774 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff92702-322b-4552-bf78-24e5d0ea6652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:59.379434 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-a9dc6277-a454-48d3-8c92-e1e66db6c86b None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:41:59.381759 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-a9dc6277-a454-48d3-8c92-e1e66db6c86b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:59.405401 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-4ff92702-322b-4552-bf78-24e5d0ea6652 None None] MFA Rules not processed for user `c36a5eaf154c463b965e5cd85ce2ab8c`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:41:59.406626 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-4ff92702-322b-4552-bf78-24e5d0ea6652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:59.493200 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1029/2043] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:41:59 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:41:59.498661 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff92702-322b-4552-bf78-24e5d0ea6652 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:59.500912 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-4ff92702-322b-4552-bf78-24e5d0ea6652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:59.530802 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1015/2044] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:41:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:41:59.533554 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a806671-9fdc-4d32-90b2-05e697cddd19 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:59.534146 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a806671-9fdc-4d32-90b2-05e697cddd19 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:59.534729 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a806671-9fdc-4d32-90b2-05e697cddd19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:59.571819 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-7a806671-9fdc-4d32-90b2-05e697cddd19 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:41:59.576918 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-7a806671-9fdc-4d32-90b2-05e697cddd19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:59.596470 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1016/2045] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:41:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:41:59.598631 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a806671-9fdc-4d32-90b2-05e697cddd19 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:59.612019 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff92702-322b-4552-bf78-24e5d0ea6652 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:59.617561 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a806671-9fdc-4d32-90b2-05e697cddd19 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:59.621370 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92523bfe-2ffc-4e3a-bbfc-e5cd79a390ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:59.622318 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92523bfe-2ffc-4e3a-bbfc-e5cd79a390ee admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:59.623047 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92523bfe-2ffc-4e3a-bbfc-e5cd79a390ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:59.623475 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92523bfe-2ffc-4e3a-bbfc-e5cd79a390ee admin admin] PATH_INFO: `/v3/projects` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:59.624495 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92523bfe-2ffc-4e3a-bbfc-e5cd79a390ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:59.624990 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760816d3-5ceb-45af-8218-468892f42b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:59.625928 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760816d3-5ceb-45af-8218-468892f42b89 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:59.626348 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760816d3-5ceb-45af-8218-468892f42b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:59.626783 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760816d3-5ceb-45af-8218-468892f42b89 admin admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:59.627245 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92523bfe-2ffc-4e3a-bbfc-e5cd79a390ee admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:59.628313 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760816d3-5ceb-45af-8218-468892f42b89 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:41:59.628313 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760816d3-5ceb-45af-8218-468892f42b89 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:59.631272 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760816d3-5ceb-45af-8218-468892f42b89 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:59.651263 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1017/2046] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 24 06:41:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:41:59.656562 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760816d3-5ceb-45af-8218-468892f42b89 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:59.669491 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760816d3-5ceb-45af-8218-468892f42b89 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:59.674784 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19b24f1-b745-454e-b86f-fcda21cba40d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:59.675396 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19b24f1-b745-454e-b86f-fcda21cba40d admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:59.675687 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19b24f1-b745-454e-b86f-fcda21cba40d admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:59.675944 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19b24f1-b745-454e-b86f-fcda21cba40d admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:59.676569 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19b24f1-b745-454e-b86f-fcda21cba40d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:59.678511 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19b24f1-b745-454e-b86f-fcda21cba40d admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:59.774891 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1030/2047] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:41:59 2022] POST /identity/v3/projects => generated 401 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:41:59.776415 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92523bfe-2ffc-4e3a-bbfc-e5cd79a390ee admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:59.780305 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-92523bfe-2ffc-4e3a-bbfc-e5cd79a390ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:41:59.815950 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1018/2048] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:41:59 2022] POST /identity/v3/projects => generated 383 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:41:59.819272 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19b24f1-b745-454e-b86f-fcda21cba40d admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:41:59.868388 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92523bfe-2ffc-4e3a-bbfc-e5cd79a390ee admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:59.913038 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19b24f1-b745-454e-b86f-fcda21cba40d admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:41:59.939346 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7370bb-a2ee-4dee-8687-933e6c1094e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:59.939346 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7370bb-a2ee-4dee-8687-933e6c1094e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:59.939346 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7370bb-a2ee-4dee-8687-933e6c1094e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:59.939346 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7370bb-a2ee-4dee-8687-933e6c1094e9 admin admin] PATH_INFO: `/v3/projects/dc3b812313284d40884e1bd0d21f7575` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:59.939730 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7370bb-a2ee-4dee-8687-933e6c1094e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc3b812313284d40884e1bd0d21f7575)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:59.941449 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7370bb-a2ee-4dee-8687-933e6c1094e9 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:41:59.944798 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d81b4e5-e530-4f93-b8ae-81ed443a0755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:41:59.945622 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d81b4e5-e530-4f93-b8ae-81ed443a0755 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:41:59.945909 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d81b4e5-e530-4f93-b8ae-81ed443a0755 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:41:59.946210 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d81b4e5-e530-4f93-b8ae-81ed443a0755 admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:41:59.946888 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d81b4e5-e530-4f93-b8ae-81ed443a0755 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:41:59.949697 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d81b4e5-e530-4f93-b8ae-81ed443a0755 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:00.090849 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-cf7370bb-a2ee-4dee-8687-933e6c1094e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc3b812313284d40884e1bd0d21f7575', 'request_id': 'req-cf7370bb-a2ee-4dee-8687-933e6c1094e9'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:00.099945 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:42:00.111088 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1019/2049] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:41:59 2022] POST /identity/v3/users => generated 307 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:42:00.111088 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d81b4e5-e530-4f93-b8ae-81ed443a0755 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:00.123825 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d81b4e5-e530-4f93-b8ae-81ed443a0755 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:00.195609 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee41c5e-48a8-4222-9db7-1921c84e6ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:00.196247 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee41c5e-48a8-4222-9db7-1921c84e6ab3 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:00.196538 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee41c5e-48a8-4222-9db7-1921c84e6ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:00.196789 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee41c5e-48a8-4222-9db7-1921c84e6ab3 admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:00.197464 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee41c5e-48a8-4222-9db7-1921c84e6ab3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:00.197644 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-cf7370bb-a2ee-4dee-8687-933e6c1094e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc3b812313284d40884e1bd0d21f7575', 'request_id': 'req-cf7370bb-a2ee-4dee-8687-933e6c1094e9'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:00.199743 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee41c5e-48a8-4222-9db7-1921c84e6ab3 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:00.207255 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-cf7370bb-a2ee-4dee-8687-933e6c1094e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc3b812313284d40884e1bd0d21f7575', 'request_id': 'req-cf7370bb-a2ee-4dee-8687-933e6c1094e9'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:00.248903 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-cf7370bb-a2ee-4dee-8687-933e6c1094e9 admin admin] The token cache is being invalidate because project dc3b812313284d40884e1bd0d21f7575 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81723) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:42:00.249241 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-cf7370bb-a2ee-4dee-8687-933e6c1094e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:00.250181 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-cf7370bb-a2ee-4dee-8687-933e6c1094e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:00.252319 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1031/2050] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:41:59 2022] DELETE /identity/v3/projects/dc3b812313284d40884e1bd0d21f7575 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:42:00.253296 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7370bb-a2ee-4dee-8687-933e6c1094e9 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:00.256231 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-cf7370bb-a2ee-4dee-8687-933e6c1094e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:00.270518 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7370bb-a2ee-4dee-8687-933e6c1094e9 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:00.275866 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49ea95e-91e2-4f12-b723-612cb5dac4d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:00.277083 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ea95e-91e2-4f12-b723-612cb5dac4d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:00.277083 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ea95e-91e2-4f12-b723-612cb5dac4d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:00.277083 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ea95e-91e2-4f12-b723-612cb5dac4d7 admin admin] PATH_INFO: `/v3/projects/7bf6c50b3a7a446ba5e001bee89b7595` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:00.280745 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49ea95e-91e2-4f12-b723-612cb5dac4d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7bf6c50b3a7a446ba5e001bee89b7595)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:00.283222 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49ea95e-91e2-4f12-b723-612cb5dac4d7 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:00.296473 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1020/2051] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:42:00 2022] POST /identity/v3/users => generated 292 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:42:00.298196 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee41c5e-48a8-4222-9db7-1921c84e6ab3 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:00.301302 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-2ee41c5e-48a8-4222-9db7-1921c84e6ab3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:00.324693 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee41c5e-48a8-4222-9db7-1921c84e6ab3 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:00.337588 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e79e5d2-7229-43af-87b2-53cbe4aae984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:00.338334 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e79e5d2-7229-43af-87b2-53cbe4aae984 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:00.342131 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e79e5d2-7229-43af-87b2-53cbe4aae984 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:00.342516 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e79e5d2-7229-43af-87b2-53cbe4aae984 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:00.343232 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e79e5d2-7229-43af-87b2-53cbe4aae984 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:00.344923 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e79e5d2-7229-43af-87b2-53cbe4aae984 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:00.358562 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1021/2052] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:42:00 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:42:00.362940 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e79e5d2-7229-43af-87b2-53cbe4aae984 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:00.362940 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-4e79e5d2-7229-43af-87b2-53cbe4aae984 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:00.376951 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e79e5d2-7229-43af-87b2-53cbe4aae984 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:00.382128 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63026a3-bedb-4518-8fd0-945649a2022b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:00.382814 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63026a3-bedb-4518-8fd0-945649a2022b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:00.383113 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63026a3-bedb-4518-8fd0-945649a2022b admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:00.383541 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63026a3-bedb-4518-8fd0-945649a2022b admin admin] PATH_INFO: `/v3/users/591b523735cc4f64a2215d634810eb5b` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:00.384782 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63026a3-bedb-4518-8fd0-945649a2022b admin admin] RBAC: Authorizing `identity:delete_user(user_id=591b523735cc4f64a2215d634810eb5b)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:00.387564 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63026a3-bedb-4518-8fd0-945649a2022b admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:00.426503 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-c49ea95e-91e2-4f12-b723-612cb5dac4d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7bf6c50b3a7a446ba5e001bee89b7595', 'request_id': 'req-c49ea95e-91e2-4f12-b723-612cb5dac4d7'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:00.539575 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-c49ea95e-91e2-4f12-b723-612cb5dac4d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7bf6c50b3a7a446ba5e001bee89b7595', 'request_id': 'req-c49ea95e-91e2-4f12-b723-612cb5dac4d7'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:00.548730 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-c49ea95e-91e2-4f12-b723-612cb5dac4d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7bf6c50b3a7a446ba5e001bee89b7595', 'request_id': 'req-c49ea95e-91e2-4f12-b723-612cb5dac4d7'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:00.563389 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-c49ea95e-91e2-4f12-b723-612cb5dac4d7 admin admin] The token cache is being invalidate because project 7bf6c50b3a7a446ba5e001bee89b7595 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81723) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:42:00.563389 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-c49ea95e-91e2-4f12-b723-612cb5dac4d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:00.564360 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-c49ea95e-91e2-4f12-b723-612cb5dac4d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:00.565698 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1032/2053] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:42:00 2022] DELETE /identity/v3/projects/7bf6c50b3a7a446ba5e001bee89b7595 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:42:00.567108 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49ea95e-91e2-4f12-b723-612cb5dac4d7 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:00.568711 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-c49ea95e-91e2-4f12-b723-612cb5dac4d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:00.625631 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-d63026a3-bedb-4518-8fd0-945649a2022b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '591b523735cc4f64a2215d634810eb5b'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:00.626669 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-d63026a3-bedb-4518-8fd0-945649a2022b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '591b523735cc4f64a2215d634810eb5b'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:00.627299 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-d63026a3-bedb-4518-8fd0-945649a2022b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '591b523735cc4f64a2215d634810eb5b'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:00.643523 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49ea95e-91e2-4f12-b723-612cb5dac4d7 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:00.646583 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-d63026a3-bedb-4518-8fd0-945649a2022b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '591b523735cc4f64a2215d634810eb5b'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:00.648518 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d316f4-322f-474d-ab7a-320fe7cceac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:00.649698 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d316f4-322f-474d-ab7a-320fe7cceac7 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:00.649698 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d316f4-322f-474d-ab7a-320fe7cceac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:00.649944 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d316f4-322f-474d-ab7a-320fe7cceac7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:00.651754 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d316f4-322f-474d-ab7a-320fe7cceac7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:00.653300 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d316f4-322f-474d-ab7a-320fe7cceac7 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:00.663997 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1033/2054] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:42:00 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:42:00.664832 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d316f4-322f-474d-ab7a-320fe7cceac7 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:00.668122 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-64d316f4-322f-474d-ab7a-320fe7cceac7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:00.681471 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d316f4-322f-474d-ab7a-320fe7cceac7 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:00.688295 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b25840e-71c2-4cde-bafb-4ccaf283ac6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:00.689070 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b25840e-71c2-4cde-bafb-4ccaf283ac6a admin admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:00.689387 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b25840e-71c2-4cde-bafb-4ccaf283ac6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:00.689670 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b25840e-71c2-4cde-bafb-4ccaf283ac6a admin admin] PATH_INFO: `/v3/projects/ebcfdd05ec984471bb90a257ea615c83/users/eeb4f725854140ea8ee736f3ac5f1967/roles/486144ad1bcd417e86acdbc1d14327bf` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:00.709963 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b25840e-71c2-4cde-bafb-4ccaf283ac6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebcfdd05ec984471bb90a257ea615c83, user_id=eeb4f725854140ea8ee736f3ac5f1967, role_id=486144ad1bcd417e86acdbc1d14327bf)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:00.712872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b25840e-71c2-4cde-bafb-4ccaf283ac6a admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:00.769052 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-d63026a3-bedb-4518-8fd0-945649a2022b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '591b523735cc4f64a2215d634810eb5b'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:00.790192 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1034/2055] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:42:00 2022] PUT /identity/v3/projects/ebcfdd05ec984471bb90a257ea615c83/users/eeb4f725854140ea8ee736f3ac5f1967/roles/486144ad1bcd417e86acdbc1d14327bf => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:42:00.792690 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aae9217-5aa0-42fc-bd39-fea9be983ce1 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:00.793213 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aae9217-5aa0-42fc-bd39-fea9be983ce1 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:00.793566 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aae9217-5aa0-42fc-bd39-fea9be983ce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:00.810585 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1022/2056] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:42:00 2022] DELETE /identity/v3/users/591b523735cc4f64a2215d634810eb5b => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:42:00.810585 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63026a3-bedb-4518-8fd0-945649a2022b admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:00.818298 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-d63026a3-bedb-4518-8fd0-945649a2022b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:00.832653 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-0aae9217-5aa0-42fc-bd39-fea9be983ce1 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:42:00.834207 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-0aae9217-5aa0-42fc-bd39-fea9be983ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:00.925582 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1035/2057] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:42:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:42:00.925582 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aae9217-5aa0-42fc-bd39-fea9be983ce1 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:00.937878 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aae9217-5aa0-42fc-bd39-fea9be983ce1 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:00.947312 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745bb850-b1d8-4163-8a1c-703ff5756833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:00.949101 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745bb850-b1d8-4163-8a1c-703ff5756833 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:00.949101 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745bb850-b1d8-4163-8a1c-703ff5756833 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:00.949348 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745bb850-b1d8-4163-8a1c-703ff5756833 admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:00.950624 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745bb850-b1d8-4163-8a1c-703ff5756833 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:00.952948 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745bb850-b1d8-4163-8a1c-703ff5756833 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:00.973523 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1036/2058] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:42:00 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:42:00.976334 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dce85f-dc0f-49fa-8f50-d9092439f64d None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:00.976814 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dce85f-dc0f-49fa-8f50-d9092439f64d None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:00.977354 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dce85f-dc0f-49fa-8f50-d9092439f64d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:01.013315 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-e1dce85f-dc0f-49fa-8f50-d9092439f64d None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:42:01.014308 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63026a3-bedb-4518-8fd0-945649a2022b admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:01.015181 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-e1dce85f-dc0f-49fa-8f50-d9092439f64d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:01.022137 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee48b53-d254-49ad-8d81-73c238d0d6b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:01.023896 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee48b53-d254-49ad-8d81-73c238d0d6b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:01.024392 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee48b53-d254-49ad-8d81-73c238d0d6b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:01.024877 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee48b53-d254-49ad-8d81-73c238d0d6b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:01.027123 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1037/2059] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:42:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:42:01.027287 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee48b53-d254-49ad-8d81-73c238d0d6b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:01.027816 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dce85f-dc0f-49fa-8f50-d9092439f64d None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:01.030032 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee48b53-d254-49ad-8d81-73c238d0d6b9 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:01.041154 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dce85f-dc0f-49fa-8f50-d9092439f64d None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:01.051635 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1023/2060] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:42:00 2022] GET /identity/v3/roles => generated 1420 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:42:01.051635 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee48b53-d254-49ad-8d81-73c238d0d6b9 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:01.051928 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2541dc-7703-4d7a-8ba8-f5a377d54249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:01.051928 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2541dc-7703-4d7a-8ba8-f5a377d54249 admin admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:01.051928 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2541dc-7703-4d7a-8ba8-f5a377d54249 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:01.052624 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2541dc-7703-4d7a-8ba8-f5a377d54249 admin admin] PATH_INFO: `/v3/projects/ebcfdd05ec984471bb90a257ea615c83/users/eeb4f725854140ea8ee736f3ac5f1967/roles/486144ad1bcd417e86acdbc1d14327bf` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:01.056178 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2541dc-7703-4d7a-8ba8-f5a377d54249 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebcfdd05ec984471bb90a257ea615c83, user_id=eeb4f725854140ea8ee736f3ac5f1967, role_id=486144ad1bcd417e86acdbc1d14327bf)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:01.062850 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2541dc-7703-4d7a-8ba8-f5a377d54249 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:01.064232 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee48b53-d254-49ad-8d81-73c238d0d6b9 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:01.069586 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9df7c2-7e48-4fac-8d30-852eea1ebf41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:01.070268 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9df7c2-7e48-4fac-8d30-852eea1ebf41 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:01.070724 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9df7c2-7e48-4fac-8d30-852eea1ebf41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:01.070965 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9df7c2-7e48-4fac-8d30-852eea1ebf41 admin admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:01.071793 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9df7c2-7e48-4fac-8d30-852eea1ebf41 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:42:01.072196 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9df7c2-7e48-4fac-8d30-852eea1ebf41 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:01.073952 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9df7c2-7e48-4fac-8d30-852eea1ebf41 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:01.078565 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1038/2061] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:42:01 2022] PUT /identity/v3/projects/ebcfdd05ec984471bb90a257ea615c83/users/eeb4f725854140ea8ee736f3ac5f1967/roles/486144ad1bcd417e86acdbc1d14327bf => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:42:01.079209 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2541dc-7703-4d7a-8ba8-f5a377d54249 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:01.101794 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1024/2062] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 24 06:42:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:42:01.105396 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084b6882-5aac-49d6-9318-ed9aa9403a26 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:01.105647 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084b6882-5aac-49d6-9318-ed9aa9403a26 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:01.105963 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084b6882-5aac-49d6-9318-ed9aa9403a26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:01.141395 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-084b6882-5aac-49d6-9318-ed9aa9403a26 None None] MFA Rules not processed for user `eeb4f725854140ea8ee736f3ac5f1967`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:42:01.142627 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-084b6882-5aac-49d6-9318-ed9aa9403a26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:01.193468 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2541dc-7703-4d7a-8ba8-f5a377d54249 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:01.200724 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad4e4b3-8e1e-4035-aad2-7870012aec50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:01.201608 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad4e4b3-8e1e-4035-aad2-7870012aec50 admin admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:01.201983 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad4e4b3-8e1e-4035-aad2-7870012aec50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:01.202362 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad4e4b3-8e1e-4035-aad2-7870012aec50 admin admin] PATH_INFO: `/v3/projects/84042cf72b36429fb21c444e321283bc/users/a39d54d40af24ae688cedbab4db7f6c5/roles/e1f9b167c51c4a388cbc9594db8a465d` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:01.231353 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad4e4b3-8e1e-4035-aad2-7870012aec50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84042cf72b36429fb21c444e321283bc, user_id=a39d54d40af24ae688cedbab4db7f6c5, role_id=e1f9b167c51c4a388cbc9594db8a465d)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:01.237066 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad4e4b3-8e1e-4035-aad2-7870012aec50 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:01.245239 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1025/2063] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:42:01 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:42:01.246229 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084b6882-5aac-49d6-9318-ed9aa9403a26 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:01.263754 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084b6882-5aac-49d6-9318-ed9aa9403a26 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:01.275575 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfce163-ea72-40fe-ad33-f70603a38647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:01.290859 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfce163-ea72-40fe-ad33-f70603a38647 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:01.290859 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfce163-ea72-40fe-ad33-f70603a38647 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:01.290859 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfce163-ea72-40fe-ad33-f70603a38647 admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:01.290859 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:42:01.290859 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfce163-ea72-40fe-ad33-f70603a38647 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:01.290859 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfce163-ea72-40fe-ad33-f70603a38647 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:01.370636 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1039/2064] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:42:01 2022] PUT /identity/v3/projects/84042cf72b36429fb21c444e321283bc/users/a39d54d40af24ae688cedbab4db7f6c5/roles/e1f9b167c51c4a388cbc9594db8a465d => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:42:01.371495 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad4e4b3-8e1e-4035-aad2-7870012aec50 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:01.373259 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-fad4e4b3-8e1e-4035-aad2-7870012aec50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:01.423015 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1026/2065] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:42:01 2022] POST /identity/v3/projects => generated 373 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:42:01.424643 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d839ee-38fa-47a9-ac18-eb1d2fb0d99f None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:01.425211 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d839ee-38fa-47a9-ac18-eb1d2fb0d99f None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:01.425638 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d839ee-38fa-47a9-ac18-eb1d2fb0d99f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:01.435350 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad4e4b3-8e1e-4035-aad2-7870012aec50 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:01.466437 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-86d839ee-38fa-47a9-ac18-eb1d2fb0d99f None None] MFA Rules not processed for user `a39d54d40af24ae688cedbab4db7f6c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:42:01.467664 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-86d839ee-38fa-47a9-ac18-eb1d2fb0d99f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:01.494619 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbe0acd-607b-4b86-a4f1-d9b5b0be7c55 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36639dce4f97412f89d71367f650b901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:01.495980 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbe0acd-607b-4b86-a4f1-d9b5b0be7c55 service cinder] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:01.496175 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbe0acd-607b-4b86-a4f1-d9b5b0be7c55 service cinder] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:01.496799 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbe0acd-607b-4b86-a4f1-d9b5b0be7c55 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:01.559755 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1027/2066] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:42:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:42:01.560635 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d839ee-38fa-47a9-ac18-eb1d2fb0d99f None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:01.597445 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbe0acd-607b-4b86-a4f1-d9b5b0be7c55 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:01.599362 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbe0acd-607b-4b86-a4f1-d9b5b0be7c55 service cinder] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:01.610549 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1040/2067] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:42:01 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:42:01.611941 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbe0acd-607b-4b86-a4f1-d9b5b0be7c55 service cinder] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:01.613603 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-0cbe0acd-607b-4b86-a4f1-d9b5b0be7c55 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:01.653170 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d839ee-38fa-47a9-ac18-eb1d2fb0d99f None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:01.659133 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f8540b-0efb-42a4-9876-77dba45ff236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:01.659850 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f8540b-0efb-42a4-9876-77dba45ff236 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:01.660204 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f8540b-0efb-42a4-9876-77dba45ff236 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:01.660551 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f8540b-0efb-42a4-9876-77dba45ff236 admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:01.661292 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f8540b-0efb-42a4-9876-77dba45ff236 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:01.663311 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f8540b-0efb-42a4-9876-77dba45ff236 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:01.706770 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbe0acd-607b-4b86-a4f1-d9b5b0be7c55 service cinder] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:01.711735 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a20d1a7-5886-4251-a5ae-10b5124d3de7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:01.712377 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a20d1a7-5886-4251-a5ae-10b5124d3de7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:01.712651 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a20d1a7-5886-4251-a5ae-10b5124d3de7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:01.712923 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a20d1a7-5886-4251-a5ae-10b5124d3de7 admin admin] PATH_INFO: `/v3/projects/d217f2dbe6c6409784d1275e269fd6b8` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:01.714984 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a20d1a7-5886-4251-a5ae-10b5124d3de7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d217f2dbe6c6409784d1275e269fd6b8)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:01.716909 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a20d1a7-5886-4251-a5ae-10b5124d3de7 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:01.798336 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1028/2068] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:42:01 2022] POST /identity/v3/users => generated 287 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:42:01.799437 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f8540b-0efb-42a4-9876-77dba45ff236 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:01.801175 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-72f8540b-0efb-42a4-9876-77dba45ff236 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:01.815202 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f8540b-0efb-42a4-9876-77dba45ff236 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:01.819064 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0165b369-eeb8-402f-89dc-15381a8ed63e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36639dce4f97412f89d71367f650b901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:01.819770 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0165b369-eeb8-402f-89dc-15381a8ed63e service cinder] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:01.820116 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0165b369-eeb8-402f-89dc-15381a8ed63e service cinder] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:01.820395 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0165b369-eeb8-402f-89dc-15381a8ed63e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:01.829911 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0165b369-eeb8-402f-89dc-15381a8ed63e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:01.831348 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0165b369-eeb8-402f-89dc-15381a8ed63e service cinder] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:01.840715 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1029/2069] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:42:01 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:42:01.841557 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0165b369-eeb8-402f-89dc-15381a8ed63e service cinder] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:01.855995 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0165b369-eeb8-402f-89dc-15381a8ed63e service cinder] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:01.865990 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08298f7b-e00f-462e-b61c-2e26396ff81b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:01.868518 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08298f7b-e00f-462e-b61c-2e26396ff81b admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:01.868815 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08298f7b-e00f-462e-b61c-2e26396ff81b admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:01.869310 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08298f7b-e00f-462e-b61c-2e26396ff81b admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:01.870568 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08298f7b-e00f-462e-b61c-2e26396ff81b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:01.872240 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08298f7b-e00f-462e-b61c-2e26396ff81b admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:01.884858 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-9a20d1a7-5886-4251-a5ae-10b5124d3de7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd217f2dbe6c6409784d1275e269fd6b8', 'request_id': 'req-9a20d1a7-5886-4251-a5ae-10b5124d3de7'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:01.890319 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1030/2070] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:42:01 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:42:01.891333 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08298f7b-e00f-462e-b61c-2e26396ff81b admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:01.908843 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08298f7b-e00f-462e-b61c-2e26396ff81b admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:01.965613 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff09cd17-bdc6-4ea9-9dbc-6908b7053cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:01.966260 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff09cd17-bdc6-4ea9-9dbc-6908b7053cff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:01.966539 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff09cd17-bdc6-4ea9-9dbc-6908b7053cff admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:01.972138 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff09cd17-bdc6-4ea9-9dbc-6908b7053cff admin admin] PATH_INFO: `/v3/users/a39d54d40af24ae688cedbab4db7f6c5` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:01.972138 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff09cd17-bdc6-4ea9-9dbc-6908b7053cff admin admin] RBAC: Authorizing `identity:delete_user(user_id=a39d54d40af24ae688cedbab4db7f6c5)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:01.972138 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff09cd17-bdc6-4ea9-9dbc-6908b7053cff admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:02.062928 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-9a20d1a7-5886-4251-a5ae-10b5124d3de7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd217f2dbe6c6409784d1275e269fd6b8', 'request_id': 'req-9a20d1a7-5886-4251-a5ae-10b5124d3de7'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:02.075066 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-9a20d1a7-5886-4251-a5ae-10b5124d3de7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd217f2dbe6c6409784d1275e269fd6b8', 'request_id': 'req-9a20d1a7-5886-4251-a5ae-10b5124d3de7'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:02.087119 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-9a20d1a7-5886-4251-a5ae-10b5124d3de7 admin admin] The token cache is being invalidate because project d217f2dbe6c6409784d1275e269fd6b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81723) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:42:02.087416 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-9a20d1a7-5886-4251-a5ae-10b5124d3de7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:02.088075 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-9a20d1a7-5886-4251-a5ae-10b5124d3de7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:02.090599 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1041/2071] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:42:01 2022] DELETE /identity/v3/projects/d217f2dbe6c6409784d1275e269fd6b8 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:42:02.093711 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a20d1a7-5886-4251-a5ae-10b5124d3de7 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:02.093711 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-9a20d1a7-5886-4251-a5ae-10b5124d3de7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:02.106866 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a20d1a7-5886-4251-a5ae-10b5124d3de7 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:02.112377 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e65b8e-3039-44a5-82f2-745718fc3035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:02.121272 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e65b8e-3039-44a5-82f2-745718fc3035 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:02.121272 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e65b8e-3039-44a5-82f2-745718fc3035 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:02.121272 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e65b8e-3039-44a5-82f2-745718fc3035 admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:02.121272 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e65b8e-3039-44a5-82f2-745718fc3035 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:02.121272 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e65b8e-3039-44a5-82f2-745718fc3035 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:02.126421 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1042/2072] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:42:02 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:42:02.127273 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e65b8e-3039-44a5-82f2-745718fc3035 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:02.129005 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-01e65b8e-3039-44a5-82f2-745718fc3035 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:02.197136 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e65b8e-3039-44a5-82f2-745718fc3035 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:02.201419 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e99072-5d91-4f9f-8f16-c7fb184ce566 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36639dce4f97412f89d71367f650b901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:02.206740 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e99072-5d91-4f9f-8f16-c7fb184ce566 service cinder] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:02.206740 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e99072-5d91-4f9f-8f16-c7fb184ce566 service cinder] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:02.206740 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e99072-5d91-4f9f-8f16-c7fb184ce566 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:02.206740 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-27e99072-5d91-4f9f-8f16-c7fb184ce566 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:02.279019 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e99072-5d91-4f9f-8f16-c7fb184ce566 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:02.280486 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e99072-5d91-4f9f-8f16-c7fb184ce566 service cinder] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:02.292263 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1043/2073] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:42:02 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:42:02.293819 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e99072-5d91-4f9f-8f16-c7fb184ce566 service cinder] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:02.295508 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-27e99072-5d91-4f9f-8f16-c7fb184ce566 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:02.310272 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-ff09cd17-bdc6-4ea9-9dbc-6908b7053cff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a39d54d40af24ae688cedbab4db7f6c5'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:02.310514 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e99072-5d91-4f9f-8f16-c7fb184ce566 service cinder] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:02.311891 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-ff09cd17-bdc6-4ea9-9dbc-6908b7053cff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a39d54d40af24ae688cedbab4db7f6c5'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:02.312958 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-ff09cd17-bdc6-4ea9-9dbc-6908b7053cff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a39d54d40af24ae688cedbab4db7f6c5'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:02.315922 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f616a575-bc87-4cad-90ff-b06974d726d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:02.316913 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f616a575-bc87-4cad-90ff-b06974d726d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:02.317115 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f616a575-bc87-4cad-90ff-b06974d726d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:02.317415 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f616a575-bc87-4cad-90ff-b06974d726d2 admin admin] PATH_INFO: `/v3/users/9b94e803b36144d8a255e3566e612060` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:02.318576 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f616a575-bc87-4cad-90ff-b06974d726d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b94e803b36144d8a255e3566e612060)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:02.320683 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f616a575-bc87-4cad-90ff-b06974d726d2 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:02.330067 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:42:02.331662 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-ff09cd17-bdc6-4ea9-9dbc-6908b7053cff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a39d54d40af24ae688cedbab4db7f6c5'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:02.403537 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-ff09cd17-bdc6-4ea9-9dbc-6908b7053cff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a39d54d40af24ae688cedbab4db7f6c5'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:02.432674 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1031/2074] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:42:01 2022] DELETE /identity/v3/users/a39d54d40af24ae688cedbab4db7f6c5 => generated 0 bytes in 542 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:42:02.432674 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff09cd17-bdc6-4ea9-9dbc-6908b7053cff admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:02.437071 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-ff09cd17-bdc6-4ea9-9dbc-6908b7053cff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:02.542744 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff09cd17-bdc6-4ea9-9dbc-6908b7053cff admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:02.548086 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc25f498-3955-442b-9226-58bafd9176ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:02.548757 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc25f498-3955-442b-9226-58bafd9176ed admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:02.549064 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc25f498-3955-442b-9226-58bafd9176ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:02.549453 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc25f498-3955-442b-9226-58bafd9176ed admin admin] PATH_INFO: `/v3/projects/2affaa1f6bef4cdcb559126588333d96/users/bb4a45146170497098d2071618ea6469/roles/e1f9b167c51c4a388cbc9594db8a465d` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:02.568947 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc25f498-3955-442b-9226-58bafd9176ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=2affaa1f6bef4cdcb559126588333d96, user_id=bb4a45146170497098d2071618ea6469, role_id=e1f9b167c51c4a388cbc9594db8a465d)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:02.571646 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc25f498-3955-442b-9226-58bafd9176ed admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:02.657211 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-f616a575-bc87-4cad-90ff-b06974d726d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b94e803b36144d8a255e3566e612060'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:02.657879 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-f616a575-bc87-4cad-90ff-b06974d726d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b94e803b36144d8a255e3566e612060'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:02.658475 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-f616a575-bc87-4cad-90ff-b06974d726d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b94e803b36144d8a255e3566e612060'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:02.702128 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1032/2075] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:42:02 2022] PUT /identity/v3/projects/2affaa1f6bef4cdcb559126588333d96/users/bb4a45146170497098d2071618ea6469/roles/e1f9b167c51c4a388cbc9594db8a465d => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:42:02.702128 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c24306e-dda2-422a-b422-503d02747ae7 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:02.702128 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c24306e-dda2-422a-b422-503d02747ae7 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:02.702128 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c24306e-dda2-422a-b422-503d02747ae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:02.722807 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-9c24306e-dda2-422a-b422-503d02747ae7 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:42:02.724305 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-9c24306e-dda2-422a-b422-503d02747ae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:02.772583 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-f616a575-bc87-4cad-90ff-b06974d726d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b94e803b36144d8a255e3566e612060'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:02.789650 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-f616a575-bc87-4cad-90ff-b06974d726d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b94e803b36144d8a255e3566e612060'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:02.818429 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1033/2076] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:42:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:42:02.818429 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c123f45a-3871-4d6c-baf5-e8a0295344a7 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:02.818429 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c123f45a-3871-4d6c-baf5-e8a0295344a7 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:02.818429 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c123f45a-3871-4d6c-baf5-e8a0295344a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:02.827854 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1044/2077] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:42:02 2022] DELETE /identity/v3/users/9b94e803b36144d8a255e3566e612060 => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:42:02.828790 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f616a575-bc87-4cad-90ff-b06974d726d2 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:02.830517 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-f616a575-bc87-4cad-90ff-b06974d726d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:02.841911 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-c123f45a-3871-4d6c-baf5-e8a0295344a7 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:42:02.842833 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-c123f45a-3871-4d6c-baf5-e8a0295344a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:02.933108 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f616a575-bc87-4cad-90ff-b06974d726d2 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:02.938017 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1034/2078] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:42:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:42:02.938570 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9772830-9c17-4146-a5e8-8819610127e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:02.939290 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9772830-9c17-4146-a5e8-8819610127e8 service neutron] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:02.939760 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9772830-9c17-4146-a5e8-8819610127e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:02.940463 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9772830-9c17-4146-a5e8-8819610127e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:02.949732 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c123f45a-3871-4d6c-baf5-e8a0295344a7 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:02.956407 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9772830-9c17-4146-a5e8-8819610127e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:02.957964 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9772830-9c17-4146-a5e8-8819610127e8 service neutron] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:02.960607 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c123f45a-3871-4d6c-baf5-e8a0295344a7 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:02.966198 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aaa03c-ac91-4374-bd16-c673f82960b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:02.975427 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aaa03c-ac91-4374-bd16-c673f82960b7 service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:02.975427 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aaa03c-ac91-4374-bd16-c673f82960b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:02.975427 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aaa03c-ac91-4374-bd16-c673f82960b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:02.975832 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1045/2079] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:42:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:42:02.980360 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10aaa03c-ac91-4374-bd16-c673f82960b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:02.981996 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10aaa03c-ac91-4374-bd16-c673f82960b7 service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:02.992168 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1035/2080] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:42:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:42:03.585156 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9772830-9c17-4146-a5e8-8819610127e8 service neutron] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:03.587537 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:42:03.591699 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-e9772830-9c17-4146-a5e8-8819610127e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:03.604817 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aaa03c-ac91-4374-bd16-c673f82960b7 service neutron] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:03.606712 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-10aaa03c-ac91-4374-bd16-c673f82960b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:03.627625 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aaa03c-ac91-4374-bd16-c673f82960b7 service neutron] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:03.653118 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e947cc5a-0033-4a20-b68d-c11fe7eba8f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:03.653118 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e947cc5a-0033-4a20-b68d-c11fe7eba8f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:03.653118 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e947cc5a-0033-4a20-b68d-c11fe7eba8f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:03.653637 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9772830-9c17-4146-a5e8-8819610127e8 service neutron] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:03.653816 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e947cc5a-0033-4a20-b68d-c11fe7eba8f0 admin admin] PATH_INFO: `/v3/projects/84042cf72b36429fb21c444e321283bc` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:03.657960 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e947cc5a-0033-4a20-b68d-c11fe7eba8f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84042cf72b36429fb21c444e321283bc)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:03.667352 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e947cc5a-0033-4a20-b68d-c11fe7eba8f0 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:03.685167 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bcac35-6ae4-4826-8791-4d5f920a035c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:03.693357 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bcac35-6ae4-4826-8791-4d5f920a035c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:03.693357 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bcac35-6ae4-4826-8791-4d5f920a035c admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:03.693357 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bcac35-6ae4-4826-8791-4d5f920a035c admin admin] PATH_INFO: `/v3/projects/8e61949114b44623894c127dd51a3efc` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:03.699462 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bcac35-6ae4-4826-8791-4d5f920a035c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e61949114b44623894c127dd51a3efc)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:03.716395 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bcac35-6ae4-4826-8791-4d5f920a035c admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:03.847025 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-e947cc5a-0033-4a20-b68d-c11fe7eba8f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84042cf72b36429fb21c444e321283bc', 'request_id': 'req-e947cc5a-0033-4a20-b68d-c11fe7eba8f0'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:03.856548 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:42:03.856747 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-e947cc5a-0033-4a20-b68d-c11fe7eba8f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84042cf72b36429fb21c444e321283bc', 'request_id': 'req-e947cc5a-0033-4a20-b68d-c11fe7eba8f0'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:03.898073 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-b5bcac35-6ae4-4826-8791-4d5f920a035c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e61949114b44623894c127dd51a3efc', 'request_id': 'req-b5bcac35-6ae4-4826-8791-4d5f920a035c'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:04.007870 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-e947cc5a-0033-4a20-b68d-c11fe7eba8f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84042cf72b36429fb21c444e321283bc', 'request_id': 'req-e947cc5a-0033-4a20-b68d-c11fe7eba8f0'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:04.034560 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-e947cc5a-0033-4a20-b68d-c11fe7eba8f0 admin admin] The token cache is being invalidate because project 84042cf72b36429fb21c444e321283bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:42:04.034861 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-e947cc5a-0033-4a20-b68d-c11fe7eba8f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:04.035568 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-e947cc5a-0033-4a20-b68d-c11fe7eba8f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:04.037149 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1036/2081] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:42:03 2022] DELETE /identity/v3/projects/84042cf72b36429fb21c444e321283bc => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:42:04.042912 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51251e4-9a63-4724-8b9f-0274755e6083 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:04.043197 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51251e4-9a63-4724-8b9f-0274755e6083 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:04.043933 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51251e4-9a63-4724-8b9f-0274755e6083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:04.064717 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-d51251e4-9a63-4724-8b9f-0274755e6083 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:42:04.065977 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-d51251e4-9a63-4724-8b9f-0274755e6083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:04.128712 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1037/2082] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:42:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:42:04.137344 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c37b0d2-e24f-4287-9ced-a2f8cefc9a7f None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:04.137693 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c37b0d2-e24f-4287-9ced-a2f8cefc9a7f None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:04.138640 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c37b0d2-e24f-4287-9ced-a2f8cefc9a7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:04.139938 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-b5bcac35-6ae4-4826-8791-4d5f920a035c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e61949114b44623894c127dd51a3efc', 'request_id': 'req-b5bcac35-6ae4-4826-8791-4d5f920a035c'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:04.155354 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-b5bcac35-6ae4-4826-8791-4d5f920a035c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e61949114b44623894c127dd51a3efc', 'request_id': 'req-b5bcac35-6ae4-4826-8791-4d5f920a035c'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:04.172498 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-7c37b0d2-e24f-4287-9ced-a2f8cefc9a7f None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:42:04.175372 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-7c37b0d2-e24f-4287-9ced-a2f8cefc9a7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:04.175910 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-b5bcac35-6ae4-4826-8791-4d5f920a035c admin admin] The token cache is being invalidate because project 8e61949114b44623894c127dd51a3efc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81723) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:42:04.176417 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-b5bcac35-6ae4-4826-8791-4d5f920a035c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:04.177374 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-b5bcac35-6ae4-4826-8791-4d5f920a035c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:04.179732 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1046/2083] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:42:03 2022] DELETE /identity/v3/projects/8e61949114b44623894c127dd51a3efc => generated 0 bytes in 595 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:42:04.188217 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe67c6c-bfcc-4d02-a7c1-35887414b749 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:04.188697 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe67c6c-bfcc-4d02-a7c1-35887414b749 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:04.189139 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe67c6c-bfcc-4d02-a7c1-35887414b749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:04.191086 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1038/2084] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:42:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:42:04.195242 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c37b0d2-e24f-4287-9ced-a2f8cefc9a7f None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:04.206816 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c37b0d2-e24f-4287-9ced-a2f8cefc9a7f None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:04.212554 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eac8286-6277-4885-9922-1e5a893dd96d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:04.214474 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eac8286-6277-4885-9922-1e5a893dd96d admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:04.214474 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eac8286-6277-4885-9922-1e5a893dd96d admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:04.215006 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eac8286-6277-4885-9922-1e5a893dd96d admin admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:04.216383 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eac8286-6277-4885-9922-1e5a893dd96d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:42:04.216789 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eac8286-6277-4885-9922-1e5a893dd96d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:04.218470 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eac8286-6277-4885-9922-1e5a893dd96d admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:04.227172 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-7fe67c6c-bfcc-4d02-a7c1-35887414b749 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:42:04.230340 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-7fe67c6c-bfcc-4d02-a7c1-35887414b749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:04.253251 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1047/2085] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:42:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:42:04.256008 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1039/2086] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 24 06:42:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:42:04.268735 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe67c6c-bfcc-4d02-a7c1-35887414b749 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:04.269371 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931955c2-6975-489f-acc3-324dc81b7887 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:04.270063 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931955c2-6975-489f-acc3-324dc81b7887 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:04.270743 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931955c2-6975-489f-acc3-324dc81b7887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:04.288355 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe67c6c-bfcc-4d02-a7c1-35887414b749 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:04.304228 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13554172-5a00-401b-b3fe-7bc64421a6de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:04.316174 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13554172-5a00-401b-b3fe-7bc64421a6de admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:04.316174 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13554172-5a00-401b-b3fe-7bc64421a6de admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:04.316694 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-931955c2-6975-489f-acc3-324dc81b7887 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:42:04.317080 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13554172-5a00-401b-b3fe-7bc64421a6de admin admin] PATH_INFO: `/v3/projects` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:04.317080 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13554172-5a00-401b-b3fe-7bc64421a6de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:04.317405 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-931955c2-6975-489f-acc3-324dc81b7887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:04.317556 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13554172-5a00-401b-b3fe-7bc64421a6de admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:04.328378 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1040/2087] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:42:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:42:04.337064 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931955c2-6975-489f-acc3-324dc81b7887 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:04.360948 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931955c2-6975-489f-acc3-324dc81b7887 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:04.367091 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace17006-e13c-420d-9f55-5902d6089f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:04.367714 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace17006-e13c-420d-9f55-5902d6089f1b admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:04.368059 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace17006-e13c-420d-9f55-5902d6089f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:04.368337 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace17006-e13c-420d-9f55-5902d6089f1b admin admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:04.368994 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace17006-e13c-420d-9f55-5902d6089f1b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:42:04.369872 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace17006-e13c-420d-9f55-5902d6089f1b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:04.371581 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace17006-e13c-420d-9f55-5902d6089f1b admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:04.392757 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1041/2088] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 24 06:42:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:42:04.398500 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace17006-e13c-420d-9f55-5902d6089f1b admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:04.409967 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace17006-e13c-420d-9f55-5902d6089f1b admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:04.419074 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c00f959-024b-42b6-8794-f86ebd132939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:04.420336 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c00f959-024b-42b6-8794-f86ebd132939 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:04.420792 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c00f959-024b-42b6-8794-f86ebd132939 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:04.421388 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c00f959-024b-42b6-8794-f86ebd132939 admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:04.422363 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c00f959-024b-42b6-8794-f86ebd132939 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:04.425146 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c00f959-024b-42b6-8794-f86ebd132939 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:04.445137 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1048/2089] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:42:04 2022] POST /identity/v3/projects => generated 385 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:42:04.446141 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13554172-5a00-401b-b3fe-7bc64421a6de admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:04.532941 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13554172-5a00-401b-b3fe-7bc64421a6de admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:04.539221 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee91331-3369-4a6a-ae5e-d8af9d06802c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:04.539853 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee91331-3369-4a6a-ae5e-d8af9d06802c admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:04.540140 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee91331-3369-4a6a-ae5e-d8af9d06802c admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:04.540415 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee91331-3369-4a6a-ae5e-d8af9d06802c admin admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:04.541420 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee91331-3369-4a6a-ae5e-d8af9d06802c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:04.543597 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee91331-3369-4a6a-ae5e-d8af9d06802c admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:04.561277 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1042/2090] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:42:04 2022] POST /identity/v3/projects => generated 391 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:42:04.566404 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c00f959-024b-42b6-8794-f86ebd132939 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:04.650557 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1049/2091] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:42:04 2022] POST /identity/v3/users => generated 293 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:42:04.655027 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee91331-3369-4a6a-ae5e-d8af9d06802c admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:04.667842 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee91331-3369-4a6a-ae5e-d8af9d06802c admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:04.672900 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c98caa9-b0dd-4e0b-b110-42076c0c6527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:04.673546 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c98caa9-b0dd-4e0b-b110-42076c0c6527 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:04.673829 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c98caa9-b0dd-4e0b-b110-42076c0c6527 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:04.674107 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c98caa9-b0dd-4e0b-b110-42076c0c6527 admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:04.674537 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c00f959-024b-42b6-8794-f86ebd132939 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:04.674890 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c98caa9-b0dd-4e0b-b110-42076c0c6527 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:04.676493 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c98caa9-b0dd-4e0b-b110-42076c0c6527 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:04.679914 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491341d1-c449-4f93-b0c9-0fd0d4302d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:04.680554 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491341d1-c449-4f93-b0c9-0fd0d4302d05 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:04.683297 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491341d1-c449-4f93-b0c9-0fd0d4302d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:04.683297 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491341d1-c449-4f93-b0c9-0fd0d4302d05 admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:04.683297 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491341d1-c449-4f93-b0c9-0fd0d4302d05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:04.684454 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491341d1-c449-4f93-b0c9-0fd0d4302d05 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:04.689491 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1050/2092] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:42:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:42:04.693803 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c98caa9-b0dd-4e0b-b110-42076c0c6527 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:04.705289 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c98caa9-b0dd-4e0b-b110-42076c0c6527 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:04.710138 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1869ae88-ad6a-42bf-af2a-f1c9fd00a01f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:04.710738 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1869ae88-ad6a-42bf-af2a-f1c9fd00a01f admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:04.711017 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1869ae88-ad6a-42bf-af2a-f1c9fd00a01f admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:04.711291 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1869ae88-ad6a-42bf-af2a-f1c9fd00a01f admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:04.711938 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1869ae88-ad6a-42bf-af2a-f1c9fd00a01f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:04.713742 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1869ae88-ad6a-42bf-af2a-f1c9fd00a01f admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:04.725344 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1051/2093] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:42:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:42:04.730276 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1869ae88-ad6a-42bf-af2a-f1c9fd00a01f admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:04.740230 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1869ae88-ad6a-42bf-af2a-f1c9fd00a01f admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:04.746233 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db97bb04-d1eb-4dd6-817e-a534137ad76b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:04.746912 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db97bb04-d1eb-4dd6-817e-a534137ad76b admin admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:04.747198 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db97bb04-d1eb-4dd6-817e-a534137ad76b admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:04.747486 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db97bb04-d1eb-4dd6-817e-a534137ad76b admin admin] PATH_INFO: `/v3/projects/c7e1b566b98441abafb312ff8178bf59/users/b3c47dc03d9b446c95e7fe8f9862204b/roles/e1f9b167c51c4a388cbc9594db8a465d` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:04.764018 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db97bb04-d1eb-4dd6-817e-a534137ad76b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7e1b566b98441abafb312ff8178bf59, user_id=b3c47dc03d9b446c95e7fe8f9862204b, role_id=e1f9b167c51c4a388cbc9594db8a465d)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:04.766655 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db97bb04-d1eb-4dd6-817e-a534137ad76b admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:04.852449 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1043/2094] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:42:04 2022] POST /identity/v3/users => generated 296 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:42:04.857712 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491341d1-c449-4f93-b0c9-0fd0d4302d05 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:04.871890 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491341d1-c449-4f93-b0c9-0fd0d4302d05 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:04.878877 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1052/2095] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:42:04 2022] PUT /identity/v3/projects/c7e1b566b98441abafb312ff8178bf59/users/b3c47dc03d9b446c95e7fe8f9862204b/roles/e1f9b167c51c4a388cbc9594db8a465d => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:42:04.892867 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291e4611-11c1-44b8-9cdc-f2541189c258 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:04.893163 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291e4611-11c1-44b8-9cdc-f2541189c258 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:04.893465 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291e4611-11c1-44b8-9cdc-f2541189c258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:04.934015 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-291e4611-11c1-44b8-9cdc-f2541189c258 None None] MFA Rules not processed for user `b3c47dc03d9b446c95e7fe8f9862204b`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:42:04.935423 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-291e4611-11c1-44b8-9cdc-f2541189c258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:04.985088 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc663590-5165-4664-9d92-28c4015b436f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:04.985836 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc663590-5165-4664-9d92-28c4015b436f admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:04.986119 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc663590-5165-4664-9d92-28c4015b436f admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:04.986419 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc663590-5165-4664-9d92-28c4015b436f admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:04.987081 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc663590-5165-4664-9d92-28c4015b436f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:04.988781 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc663590-5165-4664-9d92-28c4015b436f admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:05.001956 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1044/2096] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:42:04 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:42:05.005916 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc663590-5165-4664-9d92-28c4015b436f admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:05.019188 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc663590-5165-4664-9d92-28c4015b436f admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:05.029791 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be50354b-c615-4ac7-be05-c0a915a329ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:05.030592 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be50354b-c615-4ac7-be05-c0a915a329ce admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:05.031029 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be50354b-c615-4ac7-be05-c0a915a329ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:05.031450 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be50354b-c615-4ac7-be05-c0a915a329ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:05.031961 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be50354b-c615-4ac7-be05-c0a915a329ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:05.033976 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be50354b-c615-4ac7-be05-c0a915a329ce admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:05.042909 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1053/2097] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:42:04 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:42:05.048605 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291e4611-11c1-44b8-9cdc-f2541189c258 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:05.049309 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1045/2098] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:42:05 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:42:05.054478 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be50354b-c615-4ac7-be05-c0a915a329ce admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:05.060146 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291e4611-11c1-44b8-9cdc-f2541189c258 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:05.065575 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be50354b-c615-4ac7-be05-c0a915a329ce admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:05.074240 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d2d1f0-0048-4a48-93d2-d984d080e527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:05.078399 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d2d1f0-0048-4a48-93d2-d984d080e527 admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:05.078399 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d2d1f0-0048-4a48-93d2-d984d080e527 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:05.078399 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d2d1f0-0048-4a48-93d2-d984d080e527 admin admin] PATH_INFO: `/v3/projects` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:05.078399 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d2d1f0-0048-4a48-93d2-d984d080e527 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:05.078399 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d2d1f0-0048-4a48-93d2-d984d080e527 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:05.079201 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dd1d88-fb21-4701-9b41-c4bb12a0b1bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:05.080327 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dd1d88-fb21-4701-9b41-c4bb12a0b1bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:05.080803 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dd1d88-fb21-4701-9b41-c4bb12a0b1bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:05.081331 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dd1d88-fb21-4701-9b41-c4bb12a0b1bd admin admin] PATH_INFO: `/v3/projects/234d261de8db404f930ae38295fe6b4b/users/4904fafc137f48f191d14dde04b83600/roles/e1f9b167c51c4a388cbc9594db8a465d` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:05.101138 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dd1d88-fb21-4701-9b41-c4bb12a0b1bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=234d261de8db404f930ae38295fe6b4b, user_id=4904fafc137f48f191d14dde04b83600, role_id=e1f9b167c51c4a388cbc9594db8a465d)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:05.105433 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dd1d88-fb21-4701-9b41-c4bb12a0b1bd admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:05.280377 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1054/2099] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:42:05 2022] POST /identity/v3/projects => generated 383 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:42:05.283788 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28994927-6394-42cd-b548-1cfaee3686d0 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:05.283788 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28994927-6394-42cd-b548-1cfaee3686d0 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:05.283788 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28994927-6394-42cd-b548-1cfaee3686d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:05.310895 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-28994927-6394-42cd-b548-1cfaee3686d0 None None] MFA Rules not processed for user `bb4a45146170497098d2071618ea6469`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:42:05.312325 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-28994927-6394-42cd-b548-1cfaee3686d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:05.323235 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1046/2100] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:42:05 2022] PUT /identity/v3/projects/234d261de8db404f930ae38295fe6b4b/users/4904fafc137f48f191d14dde04b83600/roles/e1f9b167c51c4a388cbc9594db8a465d => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:42:05.333208 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dd1d88-fb21-4701-9b41-c4bb12a0b1bd admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:05.406784 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1055/2101] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:42:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:42:05.406784 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce23b38-f56c-4bee-8e54-7ec40c9a0ba5 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:05.406784 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce23b38-f56c-4bee-8e54-7ec40c9a0ba5 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:05.406784 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce23b38-f56c-4bee-8e54-7ec40c9a0ba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:05.415933 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dd1d88-fb21-4701-9b41-c4bb12a0b1bd admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:05.432130 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b32eae5-81e3-4e31-80dc-8a5f46240710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:05.432770 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b32eae5-81e3-4e31-80dc-8a5f46240710 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:05.433047 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b32eae5-81e3-4e31-80dc-8a5f46240710 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:05.433375 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b32eae5-81e3-4e31-80dc-8a5f46240710 admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:05.434535 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b32eae5-81e3-4e31-80dc-8a5f46240710 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:05.436059 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b32eae5-81e3-4e31-80dc-8a5f46240710 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:05.444165 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-bce23b38-f56c-4bee-8e54-7ec40c9a0ba5 None None] MFA Rules not processed for user `4904fafc137f48f191d14dde04b83600`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:42:05.445847 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-bce23b38-f56c-4bee-8e54-7ec40c9a0ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:05.539305 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1056/2102] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:42:05 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:42:05.539305 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce23b38-f56c-4bee-8e54-7ec40c9a0ba5 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:05.539305 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-bce23b38-f56c-4bee-8e54-7ec40c9a0ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:05.589229 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1047/2103] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:42:05 2022] POST /identity/v3/users => generated 292 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:42:05.590844 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b32eae5-81e3-4e31-80dc-8a5f46240710 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:05.593627 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-0b32eae5-81e3-4e31-80dc-8a5f46240710 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:05.624516 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce23b38-f56c-4bee-8e54-7ec40c9a0ba5 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:05.630100 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1fc23c-0b64-42e4-914b-0bab2a625ce8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36639dce4f97412f89d71367f650b901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:05.630763 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1fc23c-0b64-42e4-914b-0bab2a625ce8 service cinder] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:05.631136 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1fc23c-0b64-42e4-914b-0bab2a625ce8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:05.631426 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1fc23c-0b64-42e4-914b-0bab2a625ce8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:05.678200 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b32eae5-81e3-4e31-80dc-8a5f46240710 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:05.683843 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d977cfa-90ab-4c44-b1df-c100d924def2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36639dce4f97412f89d71367f650b901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:05.684786 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d977cfa-90ab-4c44-b1df-c100d924def2 service cinder] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:05.685122 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d977cfa-90ab-4c44-b1df-c100d924def2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:05.685445 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d977cfa-90ab-4c44-b1df-c100d924def2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:05.695496 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d977cfa-90ab-4c44-b1df-c100d924def2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:05.697964 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d977cfa-90ab-4c44-b1df-c100d924def2 service cinder] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:05.708984 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1048/2104] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:42:05 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:42:05.710206 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d977cfa-90ab-4c44-b1df-c100d924def2 service cinder] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:05.723122 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1fc23c-0b64-42e4-914b-0bab2a625ce8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:05.724256 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1fc23c-0b64-42e4-914b-0bab2a625ce8 service cinder] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:05.731332 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d977cfa-90ab-4c44-b1df-c100d924def2 service cinder] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:05.738465 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a9586f-c8f6-4348-be45-db33a71ef883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:05.740304 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a9586f-c8f6-4348-be45-db33a71ef883 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:05.740594 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a9586f-c8f6-4348-be45-db33a71ef883 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:05.740866 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a9586f-c8f6-4348-be45-db33a71ef883 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:05.741546 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a9586f-c8f6-4348-be45-db33a71ef883 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:05.743236 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a9586f-c8f6-4348-be45-db33a71ef883 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:05.744303 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1057/2105] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:42:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:42:05.759929 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1049/2106] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:42:05 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:42:05.761290 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1fc23c-0b64-42e4-914b-0bab2a625ce8 service cinder] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:05.774067 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1fc23c-0b64-42e4-914b-0bab2a625ce8 service cinder] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:05.781113 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000e72fc-65b8-4c3e-926b-c9b39feba50e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:05.783316 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000e72fc-65b8-4c3e-926b-c9b39feba50e admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:05.784130 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000e72fc-65b8-4c3e-926b-c9b39feba50e admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:05.784428 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000e72fc-65b8-4c3e-926b-c9b39feba50e admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:05.785091 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000e72fc-65b8-4c3e-926b-c9b39feba50e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:05.787247 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000e72fc-65b8-4c3e-926b-c9b39feba50e admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:05.799367 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:42:05.808828 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1058/2107] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:42:05 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:42:05.811092 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a9586f-c8f6-4348-be45-db33a71ef883 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:05.826106 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a9586f-c8f6-4348-be45-db33a71ef883 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:05.832836 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f43e837-709b-44fe-930f-5efec7fae954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:05.833809 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f43e837-709b-44fe-930f-5efec7fae954 admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:05.836334 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f43e837-709b-44fe-930f-5efec7fae954 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:05.836334 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f43e837-709b-44fe-930f-5efec7fae954 admin admin] PATH_INFO: `/v3/projects/683ec8dc43654f18834f167a90a802a5/users/61e3b8802131410696a5b2adf70bbfca/roles/e1f9b167c51c4a388cbc9594db8a465d` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:05.862079 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f43e837-709b-44fe-930f-5efec7fae954 admin admin] RBAC: Authorizing `identity:create_grant(project_id=683ec8dc43654f18834f167a90a802a5, user_id=61e3b8802131410696a5b2adf70bbfca, role_id=e1f9b167c51c4a388cbc9594db8a465d)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:05.866949 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f43e837-709b-44fe-930f-5efec7fae954 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:05.973951 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1050/2108] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:42:05 2022] PUT /identity/v3/projects/683ec8dc43654f18834f167a90a802a5/users/61e3b8802131410696a5b2adf70bbfca/roles/e1f9b167c51c4a388cbc9594db8a465d => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:42:05.985113 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4260abc9-ee51-4b5a-8984-c8980388429f None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:05.985508 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4260abc9-ee51-4b5a-8984-c8980388429f None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:05.985792 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4260abc9-ee51-4b5a-8984-c8980388429f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:06.017160 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-4260abc9-ee51-4b5a-8984-c8980388429f None None] MFA Rules not processed for user `61e3b8802131410696a5b2adf70bbfca`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:42:06.018423 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-4260abc9-ee51-4b5a-8984-c8980388429f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:06.098111 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1059/2109] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:42:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:42:06.102743 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f43e837-709b-44fe-930f-5efec7fae954 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:06.178961 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f43e837-709b-44fe-930f-5efec7fae954 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:06.186576 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09cbbbb-7927-4978-8f19-598eb0c93492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:06.187274 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09cbbbb-7927-4978-8f19-598eb0c93492 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:06.187560 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09cbbbb-7927-4978-8f19-598eb0c93492 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:06.187841 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09cbbbb-7927-4978-8f19-598eb0c93492 admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:06.188537 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09cbbbb-7927-4978-8f19-598eb0c93492 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:06.190537 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09cbbbb-7927-4978-8f19-598eb0c93492 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:06.408642 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1051/2110] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:42:06 2022] POST /identity/v3/projects => generated 385 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:42:06.409600 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4260abc9-ee51-4b5a-8984-c8980388429f None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:06.422829 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4260abc9-ee51-4b5a-8984-c8980388429f None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:06.491278 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab0fa7f-34a2-40ff-9613-e486c9b02366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:06.491934 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab0fa7f-34a2-40ff-9613-e486c9b02366 admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:06.492214 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab0fa7f-34a2-40ff-9613-e486c9b02366 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:06.492496 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab0fa7f-34a2-40ff-9613-e486c9b02366 admin admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:06.493130 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab0fa7f-34a2-40ff-9613-e486c9b02366 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:06.496057 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab0fa7f-34a2-40ff-9613-e486c9b02366 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:06.708855 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1060/2111] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:42:06 2022] POST /identity/v3/users => generated 299 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:42:06.713921 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09cbbbb-7927-4978-8f19-598eb0c93492 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:06.724870 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09cbbbb-7927-4978-8f19-598eb0c93492 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:06.731645 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694f13b1-bd25-443c-89b3-785721baae50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:06.732386 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694f13b1-bd25-443c-89b3-785721baae50 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:06.732673 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694f13b1-bd25-443c-89b3-785721baae50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:06.732976 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694f13b1-bd25-443c-89b3-785721baae50 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:06.734051 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694f13b1-bd25-443c-89b3-785721baae50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:06.735876 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694f13b1-bd25-443c-89b3-785721baae50 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:06.754808 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1052/2112] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:42:06 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:42:06.757661 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab0fa7f-34a2-40ff-9613-e486c9b02366 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:06.769611 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab0fa7f-34a2-40ff-9613-e486c9b02366 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:06.774857 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00001741-948f-4535-9a3c-f1593f24e1bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:06.776317 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00001741-948f-4535-9a3c-f1593f24e1bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:06.780767 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00001741-948f-4535-9a3c-f1593f24e1bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:06.781174 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00001741-948f-4535-9a3c-f1593f24e1bd admin admin] PATH_INFO: `/v3/projects/c3f74905a2ae47b8a3f25a42609bafa0/users/b2dc7bda390442db8eaee0c61598ff10/roles/486144ad1bcd417e86acdbc1d14327bf` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:06.801859 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00001741-948f-4535-9a3c-f1593f24e1bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3f74905a2ae47b8a3f25a42609bafa0, user_id=b2dc7bda390442db8eaee0c61598ff10, role_id=486144ad1bcd417e86acdbc1d14327bf)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:06.804466 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00001741-948f-4535-9a3c-f1593f24e1bd admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:07.096130 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1061/2113] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:42:06 2022] PUT /identity/v3/projects/c3f74905a2ae47b8a3f25a42609bafa0/users/b2dc7bda390442db8eaee0c61598ff10/roles/486144ad1bcd417e86acdbc1d14327bf => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:42:07.100305 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694f13b1-bd25-443c-89b3-785721baae50 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:07.115680 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694f13b1-bd25-443c-89b3-785721baae50 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:07.213340 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a0836d-33d1-4925-b39f-7d3153620bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:07.214015 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a0836d-33d1-4925-b39f-7d3153620bd8 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:07.214296 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a0836d-33d1-4925-b39f-7d3153620bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:07.214637 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a0836d-33d1-4925-b39f-7d3153620bd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:07.215304 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a0836d-33d1-4925-b39f-7d3153620bd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:07.216934 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a0836d-33d1-4925-b39f-7d3153620bd8 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:07.228942 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1053/2114] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:42:07 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:42:07.233687 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00001741-948f-4535-9a3c-f1593f24e1bd admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:07.247349 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00001741-948f-4535-9a3c-f1593f24e1bd admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:07.252339 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e163e59-9978-446f-b556-26d07d586faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:07.253229 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e163e59-9978-446f-b556-26d07d586faa admin admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:07.253577 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e163e59-9978-446f-b556-26d07d586faa admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:07.253914 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e163e59-9978-446f-b556-26d07d586faa admin admin] PATH_INFO: `/v3/projects/c3f74905a2ae47b8a3f25a42609bafa0/users/b2dc7bda390442db8eaee0c61598ff10/roles/486144ad1bcd417e86acdbc1d14327bf` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:07.256111 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e163e59-9978-446f-b556-26d07d586faa admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3f74905a2ae47b8a3f25a42609bafa0, user_id=b2dc7bda390442db8eaee0c61598ff10, role_id=486144ad1bcd417e86acdbc1d14327bf)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:07.259096 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e163e59-9978-446f-b556-26d07d586faa admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:07.273432 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1062/2115] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:42:07 2022] PUT /identity/v3/projects/c3f74905a2ae47b8a3f25a42609bafa0/users/b2dc7bda390442db8eaee0c61598ff10/roles/486144ad1bcd417e86acdbc1d14327bf => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:42:07.286382 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacb11a2-4c03-456e-9da6-47a1177aafa5 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:07.286977 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacb11a2-4c03-456e-9da6-47a1177aafa5 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:07.287458 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacb11a2-4c03-456e-9da6-47a1177aafa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:07.324401 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-bacb11a2-4c03-456e-9da6-47a1177aafa5 None None] MFA Rules not processed for user `b2dc7bda390442db8eaee0c61598ff10`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:42:07.327529 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-bacb11a2-4c03-456e-9da6-47a1177aafa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:07.474012 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1054/2116] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:42:07 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:42:07.490606 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e163e59-9978-446f-b556-26d07d586faa admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:07.571569 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e163e59-9978-446f-b556-26d07d586faa admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:07.578741 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47448b98-0599-4fd4-a671-feb6c53489ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36639dce4f97412f89d71367f650b901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:07.579732 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47448b98-0599-4fd4-a671-feb6c53489ea service cinder] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:07.580239 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47448b98-0599-4fd4-a671-feb6c53489ea service cinder] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:07.580695 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47448b98-0599-4fd4-a671-feb6c53489ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:07.665778 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47448b98-0599-4fd4-a671-feb6c53489ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:07.668510 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47448b98-0599-4fd4-a671-feb6c53489ea service cinder] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:07.681258 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1063/2117] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:42:07 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:42:07.686087 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:42:09.110761 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:42:09.316995 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e638b76-0966-4481-bbe5-a12e6d654cec None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:09.317449 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e638b76-0966-4481-bbe5-a12e6d654cec None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:09.317449 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e638b76-0966-4481-bbe5-a12e6d654cec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:09.346275 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-5e638b76-0966-4481-bbe5-a12e6d654cec None None] MFA Rules not processed for user `bb4a45146170497098d2071618ea6469`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:42:09.347833 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-5e638b76-0966-4481-bbe5-a12e6d654cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:09.437541 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1055/2118] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:42:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:42:10.001561 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47448b98-0599-4fd4-a671-feb6c53489ea service cinder] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:10.003546 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-47448b98-0599-4fd4-a671-feb6c53489ea service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:10.070228 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:42:10.158723 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47448b98-0599-4fd4-a671-feb6c53489ea service cinder] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:10.176595 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337d05fa-82f1-436a-9211-ee40902172f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:10.179360 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d05fa-82f1-436a-9211-ee40902172f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:10.179639 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d05fa-82f1-436a-9211-ee40902172f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:10.180815 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d05fa-82f1-436a-9211-ee40902172f6 admin admin] PATH_INFO: `/v3/users/c36a5eaf154c463b965e5cd85ce2ab8c` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:10.186271 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337d05fa-82f1-436a-9211-ee40902172f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c36a5eaf154c463b965e5cd85ce2ab8c)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:10.188242 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337d05fa-82f1-436a-9211-ee40902172f6 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:10.208557 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e638b76-0966-4481-bbe5-a12e6d654cec None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:10.251338 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e638b76-0966-4481-bbe5-a12e6d654cec None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:10.260099 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125aa95b-708f-45ac-ab22-bf5655579f94 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36639dce4f97412f89d71367f650b901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:10.261689 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125aa95b-708f-45ac-ab22-bf5655579f94 service cinder] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:10.266451 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125aa95b-708f-45ac-ab22-bf5655579f94 service cinder] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:10.266451 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125aa95b-708f-45ac-ab22-bf5655579f94 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:10.349656 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125aa95b-708f-45ac-ab22-bf5655579f94 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:10.352072 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125aa95b-708f-45ac-ab22-bf5655579f94 service cinder] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:10.363787 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1056/2119] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:42:10 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:42:10.480629 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-337d05fa-82f1-436a-9211-ee40902172f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c36a5eaf154c463b965e5cd85ce2ab8c'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:10.482208 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-337d05fa-82f1-436a-9211-ee40902172f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c36a5eaf154c463b965e5cd85ce2ab8c'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:10.484361 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-337d05fa-82f1-436a-9211-ee40902172f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c36a5eaf154c463b965e5cd85ce2ab8c'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:10.583022 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-337d05fa-82f1-436a-9211-ee40902172f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c36a5eaf154c463b965e5cd85ce2ab8c'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:10.606076 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-337d05fa-82f1-436a-9211-ee40902172f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c36a5eaf154c463b965e5cd85ce2ab8c'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:10.647020 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b804a52b-982a-48fb-a13a-ce32b7f54866 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:10.647658 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b804a52b-982a-48fb-a13a-ce32b7f54866 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:10.648201 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b804a52b-982a-48fb-a13a-ce32b7f54866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:10.653796 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1064/2120] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:42:10 2022] DELETE /identity/v3/users/c36a5eaf154c463b965e5cd85ce2ab8c => generated 0 bytes in 638 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:42:10.691816 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-b804a52b-982a-48fb-a13a-ce32b7f54866 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:42:10.693050 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-b804a52b-982a-48fb-a13a-ce32b7f54866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:10.740920 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2948bef-6b0a-4dcd-879f-0afe7c16594e None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:10.741223 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2948bef-6b0a-4dcd-879f-0afe7c16594e None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:10.741512 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2948bef-6b0a-4dcd-879f-0afe7c16594e None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:10.743084 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1065/2121] 10.222.0.47 () {54 vars in 921 bytes} [Wed Aug 24 06:42:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 06:42:10.755065 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13292fa3-1038-4de4-81dc-eeed81c78508 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:10.755353 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13292fa3-1038-4de4-81dc-eeed81c78508 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:10.755636 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13292fa3-1038-4de4-81dc-eeed81c78508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:10.792459 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1057/2122] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:42:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:42:10.803929 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-13292fa3-1038-4de4-81dc-eeed81c78508 None None] MFA Rules not processed for user `306188d42dbd4b5b84082ab3842eaf9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:42:10.806412 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-13292fa3-1038-4de4-81dc-eeed81c78508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:10.812310 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b804a52b-982a-48fb-a13a-ce32b7f54866 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:10.813768 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-b804a52b-982a-48fb-a13a-ce32b7f54866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:10.882764 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b804a52b-982a-48fb-a13a-ce32b7f54866 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:10.889831 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733e1cd1-91b3-468f-a0fb-4ae03eb94476 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:10.890601 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733e1cd1-91b3-468f-a0fb-4ae03eb94476 service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:10.890766 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733e1cd1-91b3-468f-a0fb-4ae03eb94476 service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:10.891063 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733e1cd1-91b3-468f-a0fb-4ae03eb94476 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:10.902676 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733e1cd1-91b3-468f-a0fb-4ae03eb94476 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:10.903874 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733e1cd1-91b3-468f-a0fb-4ae03eb94476 service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:10.923123 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1066/2123] 10.222.0.47 () {58 vars in 1018 bytes} [Wed Aug 24 06:42:10 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:42:10.924926 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13292fa3-1038-4de4-81dc-eeed81c78508 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:10.927218 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-13292fa3-1038-4de4-81dc-eeed81c78508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:10.936654 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1058/2124] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:42:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:42:10.953501 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13292fa3-1038-4de4-81dc-eeed81c78508 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:10.960429 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733e1cd1-91b3-468f-a0fb-4ae03eb94476 service neutron] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:10.963766 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-733e1cd1-91b3-468f-a0fb-4ae03eb94476 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:10.963994 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76e7284-ed93-468b-8787-a2626d4c7aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:10.964188 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76e7284-ed93-468b-8787-a2626d4c7aed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:10.964869 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76e7284-ed93-468b-8787-a2626d4c7aed admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:10.965366 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76e7284-ed93-468b-8787-a2626d4c7aed admin admin] PATH_INFO: `/v3/users/68b76ae7f1aa41f2acf7e18790887bd5` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:10.967248 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76e7284-ed93-468b-8787-a2626d4c7aed admin admin] RBAC: Authorizing `identity:delete_user(user_id=68b76ae7f1aa41f2acf7e18790887bd5)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:10.969221 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76e7284-ed93-468b-8787-a2626d4c7aed admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:10.985438 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733e1cd1-91b3-468f-a0fb-4ae03eb94476 service neutron] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:10.991662 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b73bd0a-0dd2-4ece-bb9c-fb13d82bbf02 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '306188d42dbd4b5b84082ab3842eaf9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:10.992289 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b73bd0a-0dd2-4ece-bb9c-fb13d82bbf02 service placement] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:10.992582 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b73bd0a-0dd2-4ece-bb9c-fb13d82bbf02 service placement] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:10.992840 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b73bd0a-0dd2-4ece-bb9c-fb13d82bbf02 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:11.004529 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b73bd0a-0dd2-4ece-bb9c-fb13d82bbf02 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:11.006835 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b73bd0a-0dd2-4ece-bb9c-fb13d82bbf02 service placement] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:11.017899 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1059/2125] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:42:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:42:11.390091 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-a76e7284-ed93-468b-8787-a2626d4c7aed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68b76ae7f1aa41f2acf7e18790887bd5'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:11.391335 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-a76e7284-ed93-468b-8787-a2626d4c7aed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68b76ae7f1aa41f2acf7e18790887bd5'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:11.392299 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-a76e7284-ed93-468b-8787-a2626d4c7aed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68b76ae7f1aa41f2acf7e18790887bd5'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:11.491617 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-a76e7284-ed93-468b-8787-a2626d4c7aed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68b76ae7f1aa41f2acf7e18790887bd5'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:11.506826 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-a76e7284-ed93-468b-8787-a2626d4c7aed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68b76ae7f1aa41f2acf7e18790887bd5'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:11.520760 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1067/2126] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:42:10 2022] DELETE /identity/v3/users/68b76ae7f1aa41f2acf7e18790887bd5 => generated 0 bytes in 597 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:42:11.574438 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b73bd0a-0dd2-4ece-bb9c-fb13d82bbf02 service placement] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:11.576118 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-3b73bd0a-0dd2-4ece-bb9c-fb13d82bbf02 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:11.595609 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b73bd0a-0dd2-4ece-bb9c-fb13d82bbf02 service placement] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:11.682241 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0235aa-3793-4f2f-ae96-8d91cbf9488e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:11.683946 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0235aa-3793-4f2f-ae96-8d91cbf9488e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:11.684545 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0235aa-3793-4f2f-ae96-8d91cbf9488e admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:11.684905 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0235aa-3793-4f2f-ae96-8d91cbf9488e admin admin] PATH_INFO: `/v3/projects/e11fa7a71be1441d9045380ae0e38fc0` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:11.687063 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0235aa-3793-4f2f-ae96-8d91cbf9488e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e11fa7a71be1441d9045380ae0e38fc0)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:11.690739 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0235aa-3793-4f2f-ae96-8d91cbf9488e admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:11.784498 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:42:11.887086 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-6c0235aa-3793-4f2f-ae96-8d91cbf9488e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e11fa7a71be1441d9045380ae0e38fc0', 'request_id': 'req-6c0235aa-3793-4f2f-ae96-8d91cbf9488e'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:11.892789 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-6c0235aa-3793-4f2f-ae96-8d91cbf9488e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e11fa7a71be1441d9045380ae0e38fc0', 'request_id': 'req-6c0235aa-3793-4f2f-ae96-8d91cbf9488e'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:11.991697 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76e7284-ed93-468b-8787-a2626d4c7aed admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:11.994364 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-a76e7284-ed93-468b-8787-a2626d4c7aed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:12.020171 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76e7284-ed93-468b-8787-a2626d4c7aed admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:12.037090 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-6c0235aa-3793-4f2f-ae96-8d91cbf9488e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e11fa7a71be1441d9045380ae0e38fc0', 'request_id': 'req-6c0235aa-3793-4f2f-ae96-8d91cbf9488e'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:12.059539 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-6c0235aa-3793-4f2f-ae96-8d91cbf9488e admin admin] The token cache is being invalidate because project e11fa7a71be1441d9045380ae0e38fc0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:42:12.059975 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-6c0235aa-3793-4f2f-ae96-8d91cbf9488e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:12.060651 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-6c0235aa-3793-4f2f-ae96-8d91cbf9488e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:12.062495 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1060/2127] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:42:11 2022] DELETE /identity/v3/projects/e11fa7a71be1441d9045380ae0e38fc0 => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:42:12.066633 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0235aa-3793-4f2f-ae96-8d91cbf9488e admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:12.068717 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-6c0235aa-3793-4f2f-ae96-8d91cbf9488e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:12.094982 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88409ad0-d5c8-4e62-a80c-78ddfad03682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:12.095622 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88409ad0-d5c8-4e62-a80c-78ddfad03682 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:12.095901 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88409ad0-d5c8-4e62-a80c-78ddfad03682 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:12.096166 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88409ad0-d5c8-4e62-a80c-78ddfad03682 admin admin] PATH_INFO: `/v3/projects/ebc642b8ee3b4946aff553abbe94c26a` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:12.097608 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88409ad0-d5c8-4e62-a80c-78ddfad03682 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebc642b8ee3b4946aff553abbe94c26a)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:12.099565 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88409ad0-d5c8-4e62-a80c-78ddfad03682 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:12.146821 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0235aa-3793-4f2f-ae96-8d91cbf9488e admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:12.155671 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d14069-ec47-4dc7-901d-3430cb4f7623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:12.155671 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d14069-ec47-4dc7-901d-3430cb4f7623 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:12.155895 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d14069-ec47-4dc7-901d-3430cb4f7623 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:12.156171 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d14069-ec47-4dc7-901d-3430cb4f7623 admin admin] PATH_INFO: `/v3/users/eeb4f725854140ea8ee736f3ac5f1967` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:12.157366 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d14069-ec47-4dc7-901d-3430cb4f7623 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eeb4f725854140ea8ee736f3ac5f1967)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:12.159306 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d14069-ec47-4dc7-901d-3430cb4f7623 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:12.231292 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-88409ad0-d5c8-4e62-a80c-78ddfad03682 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebc642b8ee3b4946aff553abbe94c26a', 'request_id': 'req-88409ad0-d5c8-4e62-a80c-78ddfad03682'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:12.339018 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-88409ad0-d5c8-4e62-a80c-78ddfad03682 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebc642b8ee3b4946aff553abbe94c26a', 'request_id': 'req-88409ad0-d5c8-4e62-a80c-78ddfad03682'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:12.351746 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-88409ad0-d5c8-4e62-a80c-78ddfad03682 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebc642b8ee3b4946aff553abbe94c26a', 'request_id': 'req-88409ad0-d5c8-4e62-a80c-78ddfad03682'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:12.362932 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-88409ad0-d5c8-4e62-a80c-78ddfad03682 admin admin] The token cache is being invalidate because project ebc642b8ee3b4946aff553abbe94c26a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81723) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:42:12.363232 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-88409ad0-d5c8-4e62-a80c-78ddfad03682 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:12.363905 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-88409ad0-d5c8-4e62-a80c-78ddfad03682 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:12.365716 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1068/2128] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:42:11 2022] DELETE /identity/v3/projects/ebc642b8ee3b4946aff553abbe94c26a => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:42:12.370496 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c80d17-4f89-45ba-aa61-99d6d7b0643b None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:12.371044 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c80d17-4f89-45ba-aa61-99d6d7b0643b None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:12.371329 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c80d17-4f89-45ba-aa61-99d6d7b0643b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:12.388859 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-28d14069-ec47-4dc7-901d-3430cb4f7623 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eeb4f725854140ea8ee736f3ac5f1967'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:12.390002 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-28d14069-ec47-4dc7-901d-3430cb4f7623 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eeb4f725854140ea8ee736f3ac5f1967'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:12.390953 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-28d14069-ec47-4dc7-901d-3430cb4f7623 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eeb4f725854140ea8ee736f3ac5f1967'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:12.406998 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-28d14069-ec47-4dc7-901d-3430cb4f7623 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eeb4f725854140ea8ee736f3ac5f1967'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:12.407364 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-42c80d17-4f89-45ba-aa61-99d6d7b0643b None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:42:12.407364 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-42c80d17-4f89-45ba-aa61-99d6d7b0643b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:12.473607 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1069/2129] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:42:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:42:12.482029 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4843342-fa1e-4973-ba25-64cd3af26880 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:12.482298 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4843342-fa1e-4973-ba25-64cd3af26880 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:12.482588 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4843342-fa1e-4973-ba25-64cd3af26880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:12.503618 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-a4843342-fa1e-4973-ba25-64cd3af26880 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:42:12.503814 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-28d14069-ec47-4dc7-901d-3430cb4f7623 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eeb4f725854140ea8ee736f3ac5f1967'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:12.504165 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-a4843342-fa1e-4973-ba25-64cd3af26880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:12.514358 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1070/2130] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:42:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:42:12.518678 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4843342-fa1e-4973-ba25-64cd3af26880 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:12.529789 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1061/2131] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:42:12 2022] DELETE /identity/v3/users/eeb4f725854140ea8ee736f3ac5f1967 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:42:12.529872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4843342-fa1e-4973-ba25-64cd3af26880 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:12.610333 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13716921-87e3-4a3e-8313-93f80b08a54f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:12.611321 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13716921-87e3-4a3e-8313-93f80b08a54f admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:12.611701 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13716921-87e3-4a3e-8313-93f80b08a54f admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:12.612104 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13716921-87e3-4a3e-8313-93f80b08a54f admin admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:12.612869 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13716921-87e3-4a3e-8313-93f80b08a54f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:42:12.613342 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13716921-87e3-4a3e-8313-93f80b08a54f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:12.615726 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13716921-87e3-4a3e-8313-93f80b08a54f admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:12.632534 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1071/2132] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 24 06:42:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:42:12.639424 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d14069-ec47-4dc7-901d-3430cb4f7623 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:12.654070 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d14069-ec47-4dc7-901d-3430cb4f7623 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:12.659622 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32660fe3-88e8-4c8d-9b4d-e56b9716b2a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:12.660556 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32660fe3-88e8-4c8d-9b4d-e56b9716b2a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:12.662133 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32660fe3-88e8-4c8d-9b4d-e56b9716b2a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:12.662133 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32660fe3-88e8-4c8d-9b4d-e56b9716b2a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:12.662133 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32660fe3-88e8-4c8d-9b4d-e56b9716b2a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:12.664075 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32660fe3-88e8-4c8d-9b4d-e56b9716b2a7 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:12.698386 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13716921-87e3-4a3e-8313-93f80b08a54f admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:12.701074 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-13716921-87e3-4a3e-8313-93f80b08a54f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:12.778922 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13716921-87e3-4a3e-8313-93f80b08a54f admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:12.787962 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404f5b48-702e-40fa-9762-f6e5949eab3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36639dce4f97412f89d71367f650b901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:12.788820 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404f5b48-702e-40fa-9762-f6e5949eab3e service cinder] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:12.789899 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404f5b48-702e-40fa-9762-f6e5949eab3e service cinder] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:12.791495 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404f5b48-702e-40fa-9762-f6e5949eab3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:12.792522 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-404f5b48-702e-40fa-9762-f6e5949eab3e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:12.801039 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1062/2133] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:42:12 2022] POST /identity/v3/projects => generated 393 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:42:12.805256 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32660fe3-88e8-4c8d-9b4d-e56b9716b2a7 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:12.899926 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404f5b48-702e-40fa-9762-f6e5949eab3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:12.900767 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32660fe3-88e8-4c8d-9b4d-e56b9716b2a7 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:12.901459 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404f5b48-702e-40fa-9762-f6e5949eab3e service cinder] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:12.906578 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6319ac36-b3ac-46df-84ce-d05a7cba291b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:12.907199 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6319ac36-b3ac-46df-84ce-d05a7cba291b admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:12.907479 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6319ac36-b3ac-46df-84ce-d05a7cba291b admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:12.907760 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6319ac36-b3ac-46df-84ce-d05a7cba291b admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:12.908436 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6319ac36-b3ac-46df-84ce-d05a7cba291b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:12.910498 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6319ac36-b3ac-46df-84ce-d05a7cba291b admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:12.915876 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1072/2134] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:42:12 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:42:13.097122 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1063/2135] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:42:12 2022] POST /identity/v3/users => generated 297 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:42:13.101559 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404f5b48-702e-40fa-9762-f6e5949eab3e service cinder] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:13.113857 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404f5b48-702e-40fa-9762-f6e5949eab3e service cinder] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:13.119296 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e86b5b9-4bb4-403a-baee-5957e1103b22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:13.119914 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e86b5b9-4bb4-403a-baee-5957e1103b22 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:13.120188 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e86b5b9-4bb4-403a-baee-5957e1103b22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:13.120464 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e86b5b9-4bb4-403a-baee-5957e1103b22 admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:13.121149 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e86b5b9-4bb4-403a-baee-5957e1103b22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:13.122771 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e86b5b9-4bb4-403a-baee-5957e1103b22 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:13.136390 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1073/2136] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:42:13 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:42:13.140103 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6319ac36-b3ac-46df-84ce-d05a7cba291b admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:13.151199 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6319ac36-b3ac-46df-84ce-d05a7cba291b admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:13.158796 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af515176-5280-49dc-a659-aff6b353352a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:13.159407 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af515176-5280-49dc-a659-aff6b353352a admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:13.159729 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af515176-5280-49dc-a659-aff6b353352a admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:13.160004 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af515176-5280-49dc-a659-aff6b353352a admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:13.160703 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af515176-5280-49dc-a659-aff6b353352a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:13.162312 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af515176-5280-49dc-a659-aff6b353352a admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:13.173218 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1064/2137] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:42:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:42:13.185071 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e86b5b9-4bb4-403a-baee-5957e1103b22 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:13.195408 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e86b5b9-4bb4-403a-baee-5957e1103b22 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:13.200103 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3321f14-6073-4e7b-adb8-3041cd8b8c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:13.200789 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3321f14-6073-4e7b-adb8-3041cd8b8c96 admin admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:13.201070 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3321f14-6073-4e7b-adb8-3041cd8b8c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:13.201324 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3321f14-6073-4e7b-adb8-3041cd8b8c96 admin admin] PATH_INFO: `/v3/projects/50896ec327454f28b1987cd4271b6e27/users/026744699ed54dce8dc240e44e8aefb3/roles/e1f9b167c51c4a388cbc9594db8a465d` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:13.217784 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3321f14-6073-4e7b-adb8-3041cd8b8c96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50896ec327454f28b1987cd4271b6e27, user_id=026744699ed54dce8dc240e44e8aefb3, role_id=e1f9b167c51c4a388cbc9594db8a465d)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:13.233144 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3321f14-6073-4e7b-adb8-3041cd8b8c96 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:13.238600 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af515176-5280-49dc-a659-aff6b353352a admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:13.240424 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-af515176-5280-49dc-a659-aff6b353352a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:13.254645 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af515176-5280-49dc-a659-aff6b353352a admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:13.259615 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71ee804-e7f6-431d-84bc-c1f1bad50ef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:13.260266 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71ee804-e7f6-431d-84bc-c1f1bad50ef5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:13.261738 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71ee804-e7f6-431d-84bc-c1f1bad50ef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:13.262018 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71ee804-e7f6-431d-84bc-c1f1bad50ef5 admin admin] PATH_INFO: `/v3/projects/ebcfdd05ec984471bb90a257ea615c83` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:13.263181 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71ee804-e7f6-431d-84bc-c1f1bad50ef5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebcfdd05ec984471bb90a257ea615c83)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:13.265098 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71ee804-e7f6-431d-84bc-c1f1bad50ef5 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:13.429123 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1074/2138] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:42:13 2022] PUT /identity/v3/projects/50896ec327454f28b1987cd4271b6e27/users/026744699ed54dce8dc240e44e8aefb3/roles/e1f9b167c51c4a388cbc9594db8a465d => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:42:13.434246 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0058e0-2193-49b3-8e26-bd4800c82da4 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:13.434534 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0058e0-2193-49b3-8e26-bd4800c82da4 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:13.434814 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0058e0-2193-49b3-8e26-bd4800c82da4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:13.453811 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-1e0058e0-2193-49b3-8e26-bd4800c82da4 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:42:13.455031 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-1e0058e0-2193-49b3-8e26-bd4800c82da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:13.476725 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:42:13.498761 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-e71ee804-e7f6-431d-84bc-c1f1bad50ef5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebcfdd05ec984471bb90a257ea615c83', 'request_id': 'req-e71ee804-e7f6-431d-84bc-c1f1bad50ef5'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:13.504718 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-e71ee804-e7f6-431d-84bc-c1f1bad50ef5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebcfdd05ec984471bb90a257ea615c83', 'request_id': 'req-e71ee804-e7f6-431d-84bc-c1f1bad50ef5'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:13.518746 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1075/2139] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:42:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:42:13.529168 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0058e0-2193-49b3-8e26-bd4800c82da4 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:13.530648 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-1e0058e0-2193-49b3-8e26-bd4800c82da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:13.590764 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0058e0-2193-49b3-8e26-bd4800c82da4 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:13.595457 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-e71ee804-e7f6-431d-84bc-c1f1bad50ef5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebcfdd05ec984471bb90a257ea615c83', 'request_id': 'req-e71ee804-e7f6-431d-84bc-c1f1bad50ef5'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:13.609915 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-e71ee804-e7f6-431d-84bc-c1f1bad50ef5 admin admin] The token cache is being invalidate because project ebcfdd05ec984471bb90a257ea615c83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:42:13.610227 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-e71ee804-e7f6-431d-84bc-c1f1bad50ef5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:13.610911 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-e71ee804-e7f6-431d-84bc-c1f1bad50ef5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:13.612441 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1065/2140] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:42:13 2022] DELETE /identity/v3/projects/ebcfdd05ec984471bb90a257ea615c83 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:42:13.619521 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b89bcff-d1d3-4dd1-8d27-f435b9f1eefe None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:13.619886 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b89bcff-d1d3-4dd1-8d27-f435b9f1eefe None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:13.620261 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b89bcff-d1d3-4dd1-8d27-f435b9f1eefe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:13.642344 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-4b89bcff-d1d3-4dd1-8d27-f435b9f1eefe None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:42:13.644328 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-4b89bcff-d1d3-4dd1-8d27-f435b9f1eefe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:13.651844 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec72ef55-50d4-4bae-a9cb-0dc9ddeba919 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:13.652476 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec72ef55-50d4-4bae-a9cb-0dc9ddeba919 service neutron] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:13.656832 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec72ef55-50d4-4bae-a9cb-0dc9ddeba919 service neutron] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:13.656832 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec72ef55-50d4-4bae-a9cb-0dc9ddeba919 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:13.658596 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-ec72ef55-50d4-4bae-a9cb-0dc9ddeba919 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:13.708326 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1066/2141] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:42:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:42:13.718896 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c96edea-a4b3-4f31-9f35-3c01a906e4ae None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:13.719189 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c96edea-a4b3-4f31-9f35-3c01a906e4ae None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:13.719490 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c96edea-a4b3-4f31-9f35-3c01a906e4ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:13.729620 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec72ef55-50d4-4bae-a9cb-0dc9ddeba919 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:13.731097 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec72ef55-50d4-4bae-a9cb-0dc9ddeba919 service neutron] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:13.739577 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-1c96edea-a4b3-4f31-9f35-3c01a906e4ae None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:42:13.740794 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-1c96edea-a4b3-4f31-9f35-3c01a906e4ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:13.741938 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1076/2142] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:42:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:42:13.749880 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1067/2143] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:42:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:42:13.754187 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec72ef55-50d4-4bae-a9cb-0dc9ddeba919 service neutron] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:13.765125 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec72ef55-50d4-4bae-a9cb-0dc9ddeba919 service neutron] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:13.770261 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59153b8-9cfd-4978-846b-967221e527a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:13.770886 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59153b8-9cfd-4978-846b-967221e527a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:13.771173 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59153b8-9cfd-4978-846b-967221e527a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:13.771452 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59153b8-9cfd-4978-846b-967221e527a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:13.772249 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59153b8-9cfd-4978-846b-967221e527a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:42:13.772560 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59153b8-9cfd-4978-846b-967221e527a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:13.774221 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59153b8-9cfd-4978-846b-967221e527a8 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:13.788793 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1077/2144] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 24 06:42:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:42:13.793770 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c96edea-a4b3-4f31-9f35-3c01a906e4ae None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:13.803043 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c96edea-a4b3-4f31-9f35-3c01a906e4ae None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:13.808102 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd183f01-9af5-4f52-b078-49588de1ddbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:13.808722 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd183f01-9af5-4f52-b078-49588de1ddbf admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:13.809003 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd183f01-9af5-4f52-b078-49588de1ddbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:13.809319 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd183f01-9af5-4f52-b078-49588de1ddbf admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:13.809970 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd183f01-9af5-4f52-b078-49588de1ddbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:13.811928 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd183f01-9af5-4f52-b078-49588de1ddbf admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:13.998662 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1068/2145] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:42:13 2022] POST /identity/v3/projects => generated 375 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:42:14.003975 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59153b8-9cfd-4978-846b-967221e527a8 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:14.015711 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59153b8-9cfd-4978-846b-967221e527a8 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:14.096997 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2d81a1-5490-4fa8-b764-c6ff98a7d668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:14.097890 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2d81a1-5490-4fa8-b764-c6ff98a7d668 admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:14.098170 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2d81a1-5490-4fa8-b764-c6ff98a7d668 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:14.098442 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2d81a1-5490-4fa8-b764-c6ff98a7d668 admin admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:14.099064 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2d81a1-5490-4fa8-b764-c6ff98a7d668 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:14.101073 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2d81a1-5490-4fa8-b764-c6ff98a7d668 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:14.238004 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1078/2146] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:42:14 2022] POST /identity/v3/users => generated 288 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:42:14.239522 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd183f01-9af5-4f52-b078-49588de1ddbf admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:14.250028 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd183f01-9af5-4f52-b078-49588de1ddbf admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:14.257318 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd939510-4ded-4358-a940-3681babb1512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:14.257952 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd939510-4ded-4358-a940-3681babb1512 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:14.258280 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd939510-4ded-4358-a940-3681babb1512 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:14.262170 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd939510-4ded-4358-a940-3681babb1512 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:14.262170 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd939510-4ded-4358-a940-3681babb1512 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:14.262170 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd939510-4ded-4358-a940-3681babb1512 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:14.281926 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1069/2147] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:42:14 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:42:14.283091 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2d81a1-5490-4fa8-b764-c6ff98a7d668 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:14.296152 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2d81a1-5490-4fa8-b764-c6ff98a7d668 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:14.300897 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63ac082-3612-4887-8652-6e46331880df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:14.301521 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63ac082-3612-4887-8652-6e46331880df admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:14.301801 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63ac082-3612-4887-8652-6e46331880df admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:14.302073 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63ac082-3612-4887-8652-6e46331880df admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:14.302736 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63ac082-3612-4887-8652-6e46331880df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:14.304319 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63ac082-3612-4887-8652-6e46331880df admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:14.316622 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1079/2148] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:42:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:42:14.327333 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd939510-4ded-4358-a940-3681babb1512 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:14.338648 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd939510-4ded-4358-a940-3681babb1512 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:14.346197 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56c5e07-8e65-4b7e-902b-85590f9b8a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:14.347082 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56c5e07-8e65-4b7e-902b-85590f9b8a34 admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:14.347818 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56c5e07-8e65-4b7e-902b-85590f9b8a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:14.348237 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56c5e07-8e65-4b7e-902b-85590f9b8a34 admin admin] PATH_INFO: `/v3/projects/654516fcd0d346f992bc04b47574f7e4/users/944abd7d819144478edafeb5d433cb45/roles/e1f9b167c51c4a388cbc9594db8a465d` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:14.375144 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56c5e07-8e65-4b7e-902b-85590f9b8a34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=654516fcd0d346f992bc04b47574f7e4, user_id=944abd7d819144478edafeb5d433cb45, role_id=e1f9b167c51c4a388cbc9594db8a465d)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:14.380372 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56c5e07-8e65-4b7e-902b-85590f9b8a34 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:14.505286 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1070/2149] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:42:14 2022] PUT /identity/v3/projects/654516fcd0d346f992bc04b47574f7e4/users/944abd7d819144478edafeb5d433cb45/roles/e1f9b167c51c4a388cbc9594db8a465d => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:42:14.524261 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b0e258-d0b7-43e2-a77a-2bdb20453f04 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:14.525588 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b0e258-d0b7-43e2-a77a-2bdb20453f04 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:14.525951 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b0e258-d0b7-43e2-a77a-2bdb20453f04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:14.560409 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-60b0e258-d0b7-43e2-a77a-2bdb20453f04 None None] MFA Rules not processed for user `944abd7d819144478edafeb5d433cb45`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:42:14.561886 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-60b0e258-d0b7-43e2-a77a-2bdb20453f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:14.643843 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1080/2150] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:42:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:42:14.653310 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56c5e07-8e65-4b7e-902b-85590f9b8a34 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:14.655660 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-a56c5e07-8e65-4b7e-902b-85590f9b8a34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:14.735811 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56c5e07-8e65-4b7e-902b-85590f9b8a34 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:14.748329 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd88462e-62bb-4a91-854f-ea9c426d30c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36639dce4f97412f89d71367f650b901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:14.748947 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd88462e-62bb-4a91-854f-ea9c426d30c1 service cinder] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:14.749250 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd88462e-62bb-4a91-854f-ea9c426d30c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:14.749526 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd88462e-62bb-4a91-854f-ea9c426d30c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:14.760352 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd88462e-62bb-4a91-854f-ea9c426d30c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:14.762045 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd88462e-62bb-4a91-854f-ea9c426d30c1 service cinder] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:14.771922 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1071/2151] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:42:14 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:42:14.945280 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:42:15.418316 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:42:17.572704 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:42:17.710206 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:42:18.784479 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:42:19.676155 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:42:20.249634 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:42:20.775002 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ad764c-da41-4513-a7da-2e2f63d6fbc7 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:20.775585 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ad764c-da41-4513-a7da-2e2f63d6fbc7 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:20.776083 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ad764c-da41-4513-a7da-2e2f63d6fbc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:20.863747 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-62ad764c-da41-4513-a7da-2e2f63d6fbc7 None None] MFA Rules not processed for user `026744699ed54dce8dc240e44e8aefb3`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:42:20.864482 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-62ad764c-da41-4513-a7da-2e2f63d6fbc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:20.975161 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1081/2152] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:42:20 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:42:20.986702 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6681ef1f-b72f-4dcc-bca9-c7aea2263eda None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:20.987620 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6681ef1f-b72f-4dcc-bca9-c7aea2263eda None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:20.987620 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6681ef1f-b72f-4dcc-bca9-c7aea2263eda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:21.008277 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-6681ef1f-b72f-4dcc-bca9-c7aea2263eda None None] MFA Rules not processed for user `026744699ed54dce8dc240e44e8aefb3`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:42:21.010094 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-6681ef1f-b72f-4dcc-bca9-c7aea2263eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:21.025595 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1072/2153] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:42:20 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:42:21.037793 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ad764c-da41-4513-a7da-2e2f63d6fbc7 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:21.039107 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-62ad764c-da41-4513-a7da-2e2f63d6fbc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:21.151275 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ad764c-da41-4513-a7da-2e2f63d6fbc7 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:21.162056 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313da14a-72c5-4451-bf38-ab0e6c6c6ed4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4f49c1b08c460f8ba6c451c9c92c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:21.162056 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313da14a-72c5-4451-bf38-ab0e6c6c6ed4 service nova] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:21.162056 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313da14a-72c5-4451-bf38-ab0e6c6c6ed4 service nova] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:21.162056 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313da14a-72c5-4451-bf38-ab0e6c6c6ed4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:21.168101 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313da14a-72c5-4451-bf38-ab0e6c6c6ed4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:21.169368 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313da14a-72c5-4451-bf38-ab0e6c6c6ed4 service nova] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:21.179224 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1082/2154] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:42:21 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:42:22.262614 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:42:22.365734 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3709ab3-2f52-477d-ae93-992ff14249bc None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:22.366659 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3709ab3-2f52-477d-ae93-992ff14249bc None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:22.367144 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3709ab3-2f52-477d-ae93-992ff14249bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:22.392978 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-b3709ab3-2f52-477d-ae93-992ff14249bc None None] MFA Rules not processed for user `026744699ed54dce8dc240e44e8aefb3`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:42:22.394702 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-b3709ab3-2f52-477d-ae93-992ff14249bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:22.404260 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1073/2155] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:42:22 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:42:22.648727 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:42:23.803245 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:42:25.382008 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:42:25.496409 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:42:26.406246 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:42:26.514581 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:42:27.046059 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:42:27.308715 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:42:28.012375 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:42:28.160369 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:42:28.403359 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313da14a-72c5-4451-bf38-ab0e6c6c6ed4 service nova] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:28.476945 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313da14a-72c5-4451-bf38-ab0e6c6c6ed4 service nova] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:28.484242 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f6381a-9dff-4ad8-81ba-3d9c3d66a2a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:28.484882 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f6381a-9dff-4ad8-81ba-3d9c3d66a2a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:28.485161 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f6381a-9dff-4ad8-81ba-3d9c3d66a2a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:28.485463 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f6381a-9dff-4ad8-81ba-3d9c3d66a2a6 admin admin] PATH_INFO: `/v3/users/944abd7d819144478edafeb5d433cb45` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:28.486614 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f6381a-9dff-4ad8-81ba-3d9c3d66a2a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=944abd7d819144478edafeb5d433cb45)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:28.488542 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f6381a-9dff-4ad8-81ba-3d9c3d66a2a6 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:28.516588 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:42:28.584694 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3709ab3-2f52-477d-ae93-992ff14249bc None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:28.586281 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-b3709ab3-2f52-477d-ae93-992ff14249bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:28.601319 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3709ab3-2f52-477d-ae93-992ff14249bc None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:28.607535 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b05356-1de9-44bb-bccc-13287f2c25ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:28.608223 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b05356-1de9-44bb-bccc-13287f2c25ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:28.608821 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b05356-1de9-44bb-bccc-13287f2c25ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:28.609086 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b05356-1de9-44bb-bccc-13287f2c25ec admin admin] PATH_INFO: `/v3/users/b3c47dc03d9b446c95e7fe8f9862204b` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:28.612687 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b05356-1de9-44bb-bccc-13287f2c25ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3c47dc03d9b446c95e7fe8f9862204b)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:28.614683 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b05356-1de9-44bb-bccc-13287f2c25ec admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:28.722304 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:42:28.736109 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-92f6381a-9dff-4ad8-81ba-3d9c3d66a2a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '944abd7d819144478edafeb5d433cb45'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:28.736769 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-92f6381a-9dff-4ad8-81ba-3d9c3d66a2a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '944abd7d819144478edafeb5d433cb45'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:28.737410 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-92f6381a-9dff-4ad8-81ba-3d9c3d66a2a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '944abd7d819144478edafeb5d433cb45'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:28.857226 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-92f6381a-9dff-4ad8-81ba-3d9c3d66a2a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '944abd7d819144478edafeb5d433cb45'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:28.873566 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-95b05356-1de9-44bb-bccc-13287f2c25ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3c47dc03d9b446c95e7fe8f9862204b'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:28.874653 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-95b05356-1de9-44bb-bccc-13287f2c25ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3c47dc03d9b446c95e7fe8f9862204b'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:28.875148 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-92f6381a-9dff-4ad8-81ba-3d9c3d66a2a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '944abd7d819144478edafeb5d433cb45'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:28.875653 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-95b05356-1de9-44bb-bccc-13287f2c25ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3c47dc03d9b446c95e7fe8f9862204b'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:28.884414 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-95b05356-1de9-44bb-bccc-13287f2c25ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3c47dc03d9b446c95e7fe8f9862204b'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:28.890526 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1083/2156] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:42:28 2022] DELETE /identity/v3/users/944abd7d819144478edafeb5d433cb45 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:42:28.894885 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af8a73e-79a7-4947-8e99-b669520acb0c None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:28.895175 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af8a73e-79a7-4947-8e99-b669520acb0c None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:28.895477 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af8a73e-79a7-4947-8e99-b669520acb0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:28.916879 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-7af8a73e-79a7-4947-8e99-b669520acb0c None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:42:28.918163 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-7af8a73e-79a7-4947-8e99-b669520acb0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:28.985221 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1084/2157] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:42:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:42:28.995769 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af8a73e-79a7-4947-8e99-b669520acb0c None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:28.997173 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-7af8a73e-79a7-4947-8e99-b669520acb0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:29.005389 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-95b05356-1de9-44bb-bccc-13287f2c25ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3c47dc03d9b446c95e7fe8f9862204b'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:29.035413 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1074/2158] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:42:28 2022] DELETE /identity/v3/users/b3c47dc03d9b446c95e7fe8f9862204b => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:42:29.037757 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28364a56-14ef-4810-b927-7f624cf4fa09 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:29.038026 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28364a56-14ef-4810-b927-7f624cf4fa09 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:29.038847 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28364a56-14ef-4810-b927-7f624cf4fa09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:29.060249 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-28364a56-14ef-4810-b927-7f624cf4fa09 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:42:29.061763 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-28364a56-14ef-4810-b927-7f624cf4fa09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:29.094914 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af8a73e-79a7-4947-8e99-b669520acb0c None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:29.157098 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1075/2159] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:42:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:42:29.169289 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28364a56-14ef-4810-b927-7f624cf4fa09 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:29.171800 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-28364a56-14ef-4810-b927-7f624cf4fa09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:29.192936 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef89ccb7-0eab-46c9-bb0f-14594dc9b5c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:29.193723 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef89ccb7-0eab-46c9-bb0f-14594dc9b5c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:29.194373 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef89ccb7-0eab-46c9-bb0f-14594dc9b5c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:29.194763 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef89ccb7-0eab-46c9-bb0f-14594dc9b5c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:29.207086 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef89ccb7-0eab-46c9-bb0f-14594dc9b5c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:29.208892 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef89ccb7-0eab-46c9-bb0f-14594dc9b5c7 service neutron] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:29.232206 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1085/2160] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:42:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:42:29.262171 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28364a56-14ef-4810-b927-7f624cf4fa09 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:29.267543 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7217c31e-9f6e-4aab-8e94-03425cc5c829 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:29.268437 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7217c31e-9f6e-4aab-8e94-03425cc5c829 service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:29.268602 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7217c31e-9f6e-4aab-8e94-03425cc5c829 service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:29.269230 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7217c31e-9f6e-4aab-8e94-03425cc5c829 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:29.283440 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7217c31e-9f6e-4aab-8e94-03425cc5c829 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:29.284860 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7217c31e-9f6e-4aab-8e94-03425cc5c829 service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:29.297507 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1076/2161] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:42:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:42:29.738970 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef89ccb7-0eab-46c9-bb0f-14594dc9b5c7 service neutron] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:29.741492 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:42:29.744330 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-ef89ccb7-0eab-46c9-bb0f-14594dc9b5c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:29.768461 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef89ccb7-0eab-46c9-bb0f-14594dc9b5c7 service neutron] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:29.777508 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a00358-60ff-458d-be90-c8f0234544ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:29.777508 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a00358-60ff-458d-be90-c8f0234544ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:29.777508 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a00358-60ff-458d-be90-c8f0234544ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:29.777508 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a00358-60ff-458d-be90-c8f0234544ac admin admin] PATH_INFO: `/v3/projects/654516fcd0d346f992bc04b47574f7e4` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:29.778407 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a00358-60ff-458d-be90-c8f0234544ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=654516fcd0d346f992bc04b47574f7e4)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:29.780710 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a00358-60ff-458d-be90-c8f0234544ac admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:29.990808 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7217c31e-9f6e-4aab-8e94-03425cc5c829 service neutron] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:29.994151 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-7217c31e-9f6e-4aab-8e94-03425cc5c829 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:30.020962 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-82a00358-60ff-458d-be90-c8f0234544ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '654516fcd0d346f992bc04b47574f7e4', 'request_id': 'req-82a00358-60ff-458d-be90-c8f0234544ac'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:30.036484 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7217c31e-9f6e-4aab-8e94-03425cc5c829 service neutron] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:30.131196 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86356a6a-807b-4efb-b624-f8e2f3ae8163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:30.132091 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86356a6a-807b-4efb-b624-f8e2f3ae8163 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:30.132371 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86356a6a-807b-4efb-b624-f8e2f3ae8163 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:30.132822 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86356a6a-807b-4efb-b624-f8e2f3ae8163 admin admin] PATH_INFO: `/v3/projects/c7e1b566b98441abafb312ff8178bf59` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:30.135046 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86356a6a-807b-4efb-b624-f8e2f3ae8163 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7e1b566b98441abafb312ff8178bf59)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:30.137154 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86356a6a-807b-4efb-b624-f8e2f3ae8163 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:30.165362 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-82a00358-60ff-458d-be90-c8f0234544ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '654516fcd0d346f992bc04b47574f7e4', 'request_id': 'req-82a00358-60ff-458d-be90-c8f0234544ac'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:30.167991 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:42:30.176412 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-82a00358-60ff-458d-be90-c8f0234544ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '654516fcd0d346f992bc04b47574f7e4', 'request_id': 'req-82a00358-60ff-458d-be90-c8f0234544ac'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:30.189650 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-82a00358-60ff-458d-be90-c8f0234544ac admin admin] The token cache is being invalidate because project 654516fcd0d346f992bc04b47574f7e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81723) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:42:30.189854 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-82a00358-60ff-458d-be90-c8f0234544ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:30.190407 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-82a00358-60ff-458d-be90-c8f0234544ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:30.191962 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1086/2162] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:42:29 2022] DELETE /identity/v3/projects/654516fcd0d346f992bc04b47574f7e4 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:42:30.197149 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248ad981-44ec-4fc8-a18c-346919925801 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:30.197465 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248ad981-44ec-4fc8-a18c-346919925801 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:30.197768 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248ad981-44ec-4fc8-a18c-346919925801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:30.218378 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-248ad981-44ec-4fc8-a18c-346919925801 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:42:30.219763 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-248ad981-44ec-4fc8-a18c-346919925801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:30.229008 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1087/2163] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:42:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:42:30.238110 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b742b294-ed2f-416d-ae39-9801af2ce286 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:30.238447 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b742b294-ed2f-416d-ae39-9801af2ce286 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:30.238733 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b742b294-ed2f-416d-ae39-9801af2ce286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:30.276333 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-b742b294-ed2f-416d-ae39-9801af2ce286 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:42:30.277759 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-b742b294-ed2f-416d-ae39-9801af2ce286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:30.285419 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-86356a6a-807b-4efb-b624-f8e2f3ae8163 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7e1b566b98441abafb312ff8178bf59', 'request_id': 'req-86356a6a-807b-4efb-b624-f8e2f3ae8163'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:30.292981 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-86356a6a-807b-4efb-b624-f8e2f3ae8163 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7e1b566b98441abafb312ff8178bf59', 'request_id': 'req-86356a6a-807b-4efb-b624-f8e2f3ae8163'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:30.362110 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1088/2164] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:42:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:42:30.366378 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b742b294-ed2f-416d-ae39-9801af2ce286 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:30.378173 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b742b294-ed2f-416d-ae39-9801af2ce286 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:30.383593 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d546792b-67a1-4a26-8781-a52c23876008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:30.384236 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d546792b-67a1-4a26-8781-a52c23876008 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:30.384521 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d546792b-67a1-4a26-8781-a52c23876008 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:30.384869 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d546792b-67a1-4a26-8781-a52c23876008 admin admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:30.385628 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d546792b-67a1-4a26-8781-a52c23876008 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:42:30.385928 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d546792b-67a1-4a26-8781-a52c23876008 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:30.387547 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d546792b-67a1-4a26-8781-a52c23876008 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:30.406176 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1089/2165] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 24 06:42:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:42:30.411036 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d546792b-67a1-4a26-8781-a52c23876008 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:30.422977 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-86356a6a-807b-4efb-b624-f8e2f3ae8163 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7e1b566b98441abafb312ff8178bf59', 'request_id': 'req-86356a6a-807b-4efb-b624-f8e2f3ae8163'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:30.425567 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d546792b-67a1-4a26-8781-a52c23876008 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:30.434885 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87913aed-2779-49f5-9fcd-5b0244d09ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:30.435727 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87913aed-2779-49f5-9fcd-5b0244d09ae2 admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:30.436012 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87913aed-2779-49f5-9fcd-5b0244d09ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:30.436288 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87913aed-2779-49f5-9fcd-5b0244d09ae2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:30.436994 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87913aed-2779-49f5-9fcd-5b0244d09ae2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:30.439582 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87913aed-2779-49f5-9fcd-5b0244d09ae2 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:30.462082 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-86356a6a-807b-4efb-b624-f8e2f3ae8163 admin admin] The token cache is being invalidate because project c7e1b566b98441abafb312ff8178bf59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:42:30.462274 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-86356a6a-807b-4efb-b624-f8e2f3ae8163 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:30.463180 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-86356a6a-807b-4efb-b624-f8e2f3ae8163 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:30.464943 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1077/2166] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:42:29 2022] DELETE /identity/v3/projects/c7e1b566b98441abafb312ff8178bf59 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:42:30.470483 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86356a6a-807b-4efb-b624-f8e2f3ae8163 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:30.472773 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-86356a6a-807b-4efb-b624-f8e2f3ae8163 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:30.491447 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86356a6a-807b-4efb-b624-f8e2f3ae8163 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:30.500606 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5199a83-7238-4146-a4b5-baa8cd6f0491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:30.502018 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5199a83-7238-4146-a4b5-baa8cd6f0491 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:30.502369 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5199a83-7238-4146-a4b5-baa8cd6f0491 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:30.502716 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5199a83-7238-4146-a4b5-baa8cd6f0491 admin admin] PATH_INFO: `/v3/users/61e3b8802131410696a5b2adf70bbfca` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:30.504019 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5199a83-7238-4146-a4b5-baa8cd6f0491 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61e3b8802131410696a5b2adf70bbfca)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:30.506174 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5199a83-7238-4146-a4b5-baa8cd6f0491 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:30.638287 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1090/2167] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:42:30 2022] POST /identity/v3/projects => generated 373 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:42:30.642911 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87913aed-2779-49f5-9fcd-5b0244d09ae2 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:30.645046 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-87913aed-2779-49f5-9fcd-5b0244d09ae2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:30.724556 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87913aed-2779-49f5-9fcd-5b0244d09ae2 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:30.732658 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4a2094-0ec6-4089-8dc9-dec3ac5768e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:30.741421 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4a2094-0ec6-4089-8dc9-dec3ac5768e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:30.741421 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4a2094-0ec6-4089-8dc9-dec3ac5768e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:30.741421 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4a2094-0ec6-4089-8dc9-dec3ac5768e8 admin admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:30.741421 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4a2094-0ec6-4089-8dc9-dec3ac5768e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:30.741421 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4a2094-0ec6-4089-8dc9-dec3ac5768e8 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:30.824886 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-f5199a83-7238-4146-a4b5-baa8cd6f0491 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61e3b8802131410696a5b2adf70bbfca'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:30.826010 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-f5199a83-7238-4146-a4b5-baa8cd6f0491 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61e3b8802131410696a5b2adf70bbfca'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:30.827792 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-f5199a83-7238-4146-a4b5-baa8cd6f0491 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61e3b8802131410696a5b2adf70bbfca'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:30.836895 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-f5199a83-7238-4146-a4b5-baa8cd6f0491 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61e3b8802131410696a5b2adf70bbfca'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:30.848321 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1091/2168] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:42:30 2022] POST /identity/v3/users => generated 287 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:42:30.852411 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4a2094-0ec6-4089-8dc9-dec3ac5768e8 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:30.854521 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-1b4a2094-0ec6-4089-8dc9-dec3ac5768e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:30.867303 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4a2094-0ec6-4089-8dc9-dec3ac5768e8 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:30.876377 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61c4eb6-e677-4d2a-aaba-be8467fdecc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:30.879081 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c4eb6-e677-4d2a-aaba-be8467fdecc1 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:30.879081 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c4eb6-e677-4d2a-aaba-be8467fdecc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:30.879081 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61c4eb6-e677-4d2a-aaba-be8467fdecc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:30.879821 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61c4eb6-e677-4d2a-aaba-be8467fdecc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:30.881611 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61c4eb6-e677-4d2a-aaba-be8467fdecc1 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:30.893386 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1092/2169] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:42:30 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:42:30.897295 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61c4eb6-e677-4d2a-aaba-be8467fdecc1 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:30.908042 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61c4eb6-e677-4d2a-aaba-be8467fdecc1 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:30.916875 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fd747e-ef32-4198-af4f-3fbdbbf3fc2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:30.917558 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fd747e-ef32-4198-af4f-3fbdbbf3fc2f admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:30.918311 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fd747e-ef32-4198-af4f-3fbdbbf3fc2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:30.918601 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fd747e-ef32-4198-af4f-3fbdbbf3fc2f admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:30.919255 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fd747e-ef32-4198-af4f-3fbdbbf3fc2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:30.921848 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fd747e-ef32-4198-af4f-3fbdbbf3fc2f admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:30.938123 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1093/2170] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:42:30 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:42:30.938967 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fd747e-ef32-4198-af4f-3fbdbbf3fc2f admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:30.949775 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fd747e-ef32-4198-af4f-3fbdbbf3fc2f admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:30.954425 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48821e8-faa9-4b7c-bdb6-ab85832d4e05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:30.955075 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48821e8-faa9-4b7c-bdb6-ab85832d4e05 admin admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:30.955354 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48821e8-faa9-4b7c-bdb6-ab85832d4e05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:30.955988 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48821e8-faa9-4b7c-bdb6-ab85832d4e05 admin admin] PATH_INFO: `/v3/projects/734d1a9636794305be833ae72a0e0409/users/a057ff6cf24a473ebb1499707e85d08e/roles/e1f9b167c51c4a388cbc9594db8a465d` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:30.965662 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-f5199a83-7238-4146-a4b5-baa8cd6f0491 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61e3b8802131410696a5b2adf70bbfca'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:30.972456 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48821e8-faa9-4b7c-bdb6-ab85832d4e05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=734d1a9636794305be833ae72a0e0409, user_id=a057ff6cf24a473ebb1499707e85d08e, role_id=e1f9b167c51c4a388cbc9594db8a465d)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:30.975652 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48821e8-faa9-4b7c-bdb6-ab85832d4e05 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:30.984961 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1078/2171] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:42:30 2022] DELETE /identity/v3/users/61e3b8802131410696a5b2adf70bbfca => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:42:31.031209 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:42:31.097211 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1094/2172] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:42:30 2022] PUT /identity/v3/projects/734d1a9636794305be833ae72a0e0409/users/a057ff6cf24a473ebb1499707e85d08e/roles/e1f9b167c51c4a388cbc9594db8a465d => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:42:31.110029 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6e274e-375b-46d1-a97d-893429d08595 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:31.110514 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6e274e-375b-46d1-a97d-893429d08595 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:31.110989 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6e274e-375b-46d1-a97d-893429d08595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:31.182851 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-2a6e274e-375b-46d1-a97d-893429d08595 None None] MFA Rules not processed for user `a057ff6cf24a473ebb1499707e85d08e`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:42:31.185335 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-2a6e274e-375b-46d1-a97d-893429d08595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:31.318326 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1079/2173] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:42:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:42:31.338943 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48821e8-faa9-4b7c-bdb6-ab85832d4e05 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:31.346672 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-b48821e8-faa9-4b7c-bdb6-ab85832d4e05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:31.373989 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:42:31.467380 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48821e8-faa9-4b7c-bdb6-ab85832d4e05 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:31.473942 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda7ac54-30c1-4e0b-aebf-29f6030fd5cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36639dce4f97412f89d71367f650b901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:31.475102 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda7ac54-30c1-4e0b-aebf-29f6030fd5cf service cinder] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:31.475534 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda7ac54-30c1-4e0b-aebf-29f6030fd5cf service cinder] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:31.475969 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda7ac54-30c1-4e0b-aebf-29f6030fd5cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:31.507090 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda7ac54-30c1-4e0b-aebf-29f6030fd5cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:31.510165 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda7ac54-30c1-4e0b-aebf-29f6030fd5cf service cinder] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:31.530360 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1095/2174] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:42:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:42:31.562745 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6e274e-375b-46d1-a97d-893429d08595 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:31.627043 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda7ac54-30c1-4e0b-aebf-29f6030fd5cf service cinder] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:31.631597 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-fda7ac54-30c1-4e0b-aebf-29f6030fd5cf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:31.677002 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6e274e-375b-46d1-a97d-893429d08595 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:31.683751 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3400bd9-45e8-420c-a292-e1542d2a7588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:31.684720 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3400bd9-45e8-420c-a292-e1542d2a7588 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:31.687670 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3400bd9-45e8-420c-a292-e1542d2a7588 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:31.688223 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3400bd9-45e8-420c-a292-e1542d2a7588 admin admin] PATH_INFO: `/v3/users/a057ff6cf24a473ebb1499707e85d08e` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:31.690069 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3400bd9-45e8-420c-a292-e1542d2a7588 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a057ff6cf24a473ebb1499707e85d08e)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:31.692411 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3400bd9-45e8-420c-a292-e1542d2a7588 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:31.772197 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:42:31.777875 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda7ac54-30c1-4e0b-aebf-29f6030fd5cf service cinder] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:31.783209 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faacde6-1ae3-4b7c-a5a0-d33a0ca11780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:31.783962 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faacde6-1ae3-4b7c-a5a0-d33a0ca11780 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:31.784261 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faacde6-1ae3-4b7c-a5a0-d33a0ca11780 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:31.784525 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faacde6-1ae3-4b7c-a5a0-d33a0ca11780 admin admin] PATH_INFO: `/v3/projects/683ec8dc43654f18834f167a90a802a5` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:31.785829 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faacde6-1ae3-4b7c-a5a0-d33a0ca11780 admin admin] RBAC: Authorizing `identity:delete_project(project_id=683ec8dc43654f18834f167a90a802a5)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:31.788046 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faacde6-1ae3-4b7c-a5a0-d33a0ca11780 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:32.004777 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-d3400bd9-45e8-420c-a292-e1542d2a7588 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a057ff6cf24a473ebb1499707e85d08e'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:32.005750 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-d3400bd9-45e8-420c-a292-e1542d2a7588 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a057ff6cf24a473ebb1499707e85d08e'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:32.006420 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-d3400bd9-45e8-420c-a292-e1542d2a7588 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a057ff6cf24a473ebb1499707e85d08e'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:32.008565 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-2faacde6-1ae3-4b7c-a5a0-d33a0ca11780 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '683ec8dc43654f18834f167a90a802a5', 'request_id': 'req-2faacde6-1ae3-4b7c-a5a0-d33a0ca11780'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:32.020231 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-d3400bd9-45e8-420c-a292-e1542d2a7588 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a057ff6cf24a473ebb1499707e85d08e'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:32.148525 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-2faacde6-1ae3-4b7c-a5a0-d33a0ca11780 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '683ec8dc43654f18834f167a90a802a5', 'request_id': 'req-2faacde6-1ae3-4b7c-a5a0-d33a0ca11780'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:32.156465 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-2faacde6-1ae3-4b7c-a5a0-d33a0ca11780 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '683ec8dc43654f18834f167a90a802a5', 'request_id': 'req-2faacde6-1ae3-4b7c-a5a0-d33a0ca11780'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:32.178594 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-2faacde6-1ae3-4b7c-a5a0-d33a0ca11780 admin admin] The token cache is being invalidate because project 683ec8dc43654f18834f167a90a802a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81723) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:42:32.178904 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-2faacde6-1ae3-4b7c-a5a0-d33a0ca11780 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:32.180345 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-2faacde6-1ae3-4b7c-a5a0-d33a0ca11780 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:32.181904 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1096/2175] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:42:31 2022] DELETE /identity/v3/projects/683ec8dc43654f18834f167a90a802a5 => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:42:32.186018 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faacde6-1ae3-4b7c-a5a0-d33a0ca11780 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:32.189470 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-2faacde6-1ae3-4b7c-a5a0-d33a0ca11780 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:32.215892 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-d3400bd9-45e8-420c-a292-e1542d2a7588 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a057ff6cf24a473ebb1499707e85d08e'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:32.244687 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1080/2176] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:42:31 2022] DELETE /identity/v3/users/a057ff6cf24a473ebb1499707e85d08e => generated 0 bytes in 679 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:42:32.251613 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146dd912-a67b-4337-a1f7-237452eb5a92 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:32.251613 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146dd912-a67b-4337-a1f7-237452eb5a92 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:32.251613 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146dd912-a67b-4337-a1f7-237452eb5a92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:32.277029 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-146dd912-a67b-4337-a1f7-237452eb5a92 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:42:32.278957 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-146dd912-a67b-4337-a1f7-237452eb5a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:32.290198 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:42:32.300623 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faacde6-1ae3-4b7c-a5a0-d33a0ca11780 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:32.395429 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6371fa-49e0-47eb-bafe-434a54010d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:32.396080 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6371fa-49e0-47eb-bafe-434a54010d40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:32.396358 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6371fa-49e0-47eb-bafe-434a54010d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:32.396690 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6371fa-49e0-47eb-bafe-434a54010d40 admin admin] PATH_INFO: `/v3/users/b2dc7bda390442db8eaee0c61598ff10` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:32.398084 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6371fa-49e0-47eb-bafe-434a54010d40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2dc7bda390442db8eaee0c61598ff10)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:32.400616 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6371fa-49e0-47eb-bafe-434a54010d40 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:32.401536 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1081/2177] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:42:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:42:32.413087 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146dd912-a67b-4337-a1f7-237452eb5a92 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:32.416257 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-146dd912-a67b-4337-a1f7-237452eb5a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:32.516930 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146dd912-a67b-4337-a1f7-237452eb5a92 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:32.529086 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089b5e6f-6a37-40a8-baec-798156cf3850 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:32.529919 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089b5e6f-6a37-40a8-baec-798156cf3850 service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:32.530294 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089b5e6f-6a37-40a8-baec-798156cf3850 service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:32.530640 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089b5e6f-6a37-40a8-baec-798156cf3850 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:32.545206 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089b5e6f-6a37-40a8-baec-798156cf3850 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:32.546933 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089b5e6f-6a37-40a8-baec-798156cf3850 service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:32.562002 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1082/2178] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:42:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:42:32.764313 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:42:32.780983 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-6f6371fa-49e0-47eb-bafe-434a54010d40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2dc7bda390442db8eaee0c61598ff10'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:32.781643 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-6f6371fa-49e0-47eb-bafe-434a54010d40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2dc7bda390442db8eaee0c61598ff10'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:32.782583 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-6f6371fa-49e0-47eb-bafe-434a54010d40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2dc7bda390442db8eaee0c61598ff10'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:32.903124 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-6f6371fa-49e0-47eb-bafe-434a54010d40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2dc7bda390442db8eaee0c61598ff10'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:32.920183 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-6f6371fa-49e0-47eb-bafe-434a54010d40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2dc7bda390442db8eaee0c61598ff10'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:32.938663 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1097/2179] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:42:32 2022] DELETE /identity/v3/users/b2dc7bda390442db8eaee0c61598ff10 => generated 0 bytes in 753 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:42:33.390042 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089b5e6f-6a37-40a8-baec-798156cf3850 service neutron] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:33.392640 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-089b5e6f-6a37-40a8-baec-798156cf3850 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:33.409444 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089b5e6f-6a37-40a8-baec-798156cf3850 service neutron] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:33.451728 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:42:33.518987 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862a639c-f66b-4013-be76-024ea8b511dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:33.519830 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862a639c-f66b-4013-be76-024ea8b511dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:33.520908 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862a639c-f66b-4013-be76-024ea8b511dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:33.521308 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862a639c-f66b-4013-be76-024ea8b511dc admin admin] PATH_INFO: `/v3/projects/734d1a9636794305be833ae72a0e0409` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:33.522870 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862a639c-f66b-4013-be76-024ea8b511dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=734d1a9636794305be833ae72a0e0409)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:33.529340 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862a639c-f66b-4013-be76-024ea8b511dc admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:33.702675 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-862a639c-f66b-4013-be76-024ea8b511dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '734d1a9636794305be833ae72a0e0409', 'request_id': 'req-862a639c-f66b-4013-be76-024ea8b511dc'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:33.718409 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-862a639c-f66b-4013-be76-024ea8b511dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '734d1a9636794305be833ae72a0e0409', 'request_id': 'req-862a639c-f66b-4013-be76-024ea8b511dc'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:33.841150 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-862a639c-f66b-4013-be76-024ea8b511dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '734d1a9636794305be833ae72a0e0409', 'request_id': 'req-862a639c-f66b-4013-be76-024ea8b511dc'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:33.850076 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6371fa-49e0-47eb-bafe-434a54010d40 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:33.851996 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-6f6371fa-49e0-47eb-bafe-434a54010d40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:33.869059 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6371fa-49e0-47eb-bafe-434a54010d40 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:33.885406 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-862a639c-f66b-4013-be76-024ea8b511dc admin admin] The token cache is being invalidate because project 734d1a9636794305be833ae72a0e0409 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:42:33.885406 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-862a639c-f66b-4013-be76-024ea8b511dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:33.885844 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-862a639c-f66b-4013-be76-024ea8b511dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:33.889441 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1083/2180] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:42:33 2022] DELETE /identity/v3/projects/734d1a9636794305be833ae72a0e0409 => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:42:33.897479 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cf76e4-6ad6-4130-84d5-2158f5d2395c None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:33.897779 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cf76e4-6ad6-4130-84d5-2158f5d2395c None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:33.898049 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cf76e4-6ad6-4130-84d5-2158f5d2395c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:33.959411 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-19cf76e4-6ad6-4130-84d5-2158f5d2395c None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:42:33.961240 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-19cf76e4-6ad6-4130-84d5-2158f5d2395c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:33.983380 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5b8557-4d17-49c6-b250-8de8330d2842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:33.983380 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5b8557-4d17-49c6-b250-8de8330d2842 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:33.983380 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5b8557-4d17-49c6-b250-8de8330d2842 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:33.983380 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5b8557-4d17-49c6-b250-8de8330d2842 admin admin] PATH_INFO: `/v3/projects/c3f74905a2ae47b8a3f25a42609bafa0` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:33.983726 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5b8557-4d17-49c6-b250-8de8330d2842 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3f74905a2ae47b8a3f25a42609bafa0)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:33.983726 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5b8557-4d17-49c6-b250-8de8330d2842 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:34.015642 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:42:34.042601 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1084/2181] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:42:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:42:34.043662 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cf76e4-6ad6-4130-84d5-2158f5d2395c None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:34.045145 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-19cf76e4-6ad6-4130-84d5-2158f5d2395c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:34.058931 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cf76e4-6ad6-4130-84d5-2158f5d2395c None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:34.064817 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96263d2f-6089-4d53-9ddb-4a66bd7ca4ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:34.065531 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96263d2f-6089-4d53-9ddb-4a66bd7ca4ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:34.065813 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96263d2f-6089-4d53-9ddb-4a66bd7ca4ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:34.066091 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96263d2f-6089-4d53-9ddb-4a66bd7ca4ca admin admin] PATH_INFO: `/v3/users/4904fafc137f48f191d14dde04b83600` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:34.067529 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96263d2f-6089-4d53-9ddb-4a66bd7ca4ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=4904fafc137f48f191d14dde04b83600)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:34.071439 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96263d2f-6089-4d53-9ddb-4a66bd7ca4ca admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:34.191307 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-fd5b8557-4d17-49c6-b250-8de8330d2842 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3f74905a2ae47b8a3f25a42609bafa0', 'request_id': 'req-fd5b8557-4d17-49c6-b250-8de8330d2842'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:34.319205 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-fd5b8557-4d17-49c6-b250-8de8330d2842 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3f74905a2ae47b8a3f25a42609bafa0', 'request_id': 'req-fd5b8557-4d17-49c6-b250-8de8330d2842'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:34.327375 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-fd5b8557-4d17-49c6-b250-8de8330d2842 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3f74905a2ae47b8a3f25a42609bafa0', 'request_id': 'req-fd5b8557-4d17-49c6-b250-8de8330d2842'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:34.340513 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-fd5b8557-4d17-49c6-b250-8de8330d2842 admin admin] The token cache is being invalidate because project c3f74905a2ae47b8a3f25a42609bafa0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81723) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:42:34.340882 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-fd5b8557-4d17-49c6-b250-8de8330d2842 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:34.341847 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-fd5b8557-4d17-49c6-b250-8de8330d2842 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:34.344026 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1098/2182] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:42:33 2022] DELETE /identity/v3/projects/c3f74905a2ae47b8a3f25a42609bafa0 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:42:34.345575 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c54700c-d379-4889-a7cd-a0b1c2a192f4 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:34.346319 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c54700c-d379-4889-a7cd-a0b1c2a192f4 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:34.346585 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c54700c-d379-4889-a7cd-a0b1c2a192f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:34.371201 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-9c54700c-d379-4889-a7cd-a0b1c2a192f4 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:42:34.372631 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-9c54700c-d379-4889-a7cd-a0b1c2a192f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:34.377933 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-96263d2f-6089-4d53-9ddb-4a66bd7ca4ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4904fafc137f48f191d14dde04b83600'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:34.381524 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-96263d2f-6089-4d53-9ddb-4a66bd7ca4ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4904fafc137f48f191d14dde04b83600'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:34.381524 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-96263d2f-6089-4d53-9ddb-4a66bd7ca4ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4904fafc137f48f191d14dde04b83600'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:34.393891 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-96263d2f-6089-4d53-9ddb-4a66bd7ca4ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4904fafc137f48f191d14dde04b83600'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:34.466013 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1099/2183] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:42:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:42:34.467582 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ff7de4-d29b-4e82-a067-a3d45598b350 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:34.467918 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ff7de4-d29b-4e82-a067-a3d45598b350 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:34.468365 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ff7de4-d29b-4e82-a067-a3d45598b350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:34.480394 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-96263d2f-6089-4d53-9ddb-4a66bd7ca4ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4904fafc137f48f191d14dde04b83600'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:34.503338 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-c8ff7de4-d29b-4e82-a067-a3d45598b350 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:42:34.504572 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-c8ff7de4-d29b-4e82-a067-a3d45598b350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:34.513800 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1085/2184] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:42:34 2022] DELETE /identity/v3/users/4904fafc137f48f191d14dde04b83600 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:42:34.515729 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96263d2f-6089-4d53-9ddb-4a66bd7ca4ca admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:34.524033 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1100/2185] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:42:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:42:34.524033 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b046c8-7d2d-47b0-8137-4f7245a63592 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:34.524033 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b046c8-7d2d-47b0-8137-4f7245a63592 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:34.524033 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b046c8-7d2d-47b0-8137-4f7245a63592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:34.542988 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-d8b046c8-7d2d-47b0-8137-4f7245a63592 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:42:34.544560 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-d8b046c8-7d2d-47b0-8137-4f7245a63592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:34.603942 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96263d2f-6089-4d53-9ddb-4a66bd7ca4ca admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:34.609157 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c72588b-779b-49e4-9166-50357e07e5e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:34.609843 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c72588b-779b-49e4-9166-50357e07e5e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:34.610159 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c72588b-779b-49e4-9166-50357e07e5e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:34.610426 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c72588b-779b-49e4-9166-50357e07e5e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:34.611087 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c72588b-779b-49e4-9166-50357e07e5e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:42:34.611398 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c72588b-779b-49e4-9166-50357e07e5e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:34.613072 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c72588b-779b-49e4-9166-50357e07e5e6 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:34.630398 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1086/2186] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 24 06:42:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:42:34.631862 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cadb6c2-f873-4369-a0f3-0b4c56762d3c None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:34.632693 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cadb6c2-f873-4369-a0f3-0b4c56762d3c None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:34.639332 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cadb6c2-f873-4369-a0f3-0b4c56762d3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:34.639566 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1101/2187] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:42:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:42:34.649405 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b046c8-7d2d-47b0-8137-4f7245a63592 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:34.661235 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b046c8-7d2d-47b0-8137-4f7245a63592 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:34.668213 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea6eaf8-fe24-4c9d-850a-6a2b6b228ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:34.668969 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea6eaf8-fe24-4c9d-850a-6a2b6b228ed7 admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:34.669175 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-8cadb6c2-f873-4369-a0f3-0b4c56762d3c None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:42:34.669409 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea6eaf8-fe24-4c9d-850a-6a2b6b228ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:34.671353 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea6eaf8-fe24-4c9d-850a-6a2b6b228ed7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:34.671353 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea6eaf8-fe24-4c9d-850a-6a2b6b228ed7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:34.671556 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-8cadb6c2-f873-4369-a0f3-0b4c56762d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:34.672667 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea6eaf8-fe24-4c9d-850a-6a2b6b228ed7 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:34.683417 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1087/2188] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:42:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:42:34.687953 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cadb6c2-f873-4369-a0f3-0b4c56762d3c None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:34.690219 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-8cadb6c2-f873-4369-a0f3-0b4c56762d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:34.765358 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cadb6c2-f873-4369-a0f3-0b4c56762d3c None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:34.772023 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1102/2189] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:42:34 2022] POST /identity/v3/projects => generated 397 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:42:34.772023 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea6eaf8-fe24-4c9d-850a-6a2b6b228ed7 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:34.830676 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86cccab-ae5c-4ea5-98ff-bb4036a2fa25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:34.831736 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86cccab-ae5c-4ea5-98ff-bb4036a2fa25 service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:34.832602 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86cccab-ae5c-4ea5-98ff-bb4036a2fa25 service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:34.833101 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86cccab-ae5c-4ea5-98ff-bb4036a2fa25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:34.884515 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea6eaf8-fe24-4c9d-850a-6a2b6b228ed7 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:34.892359 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180f6076-5909-4516-b2de-96e5e81ff4a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:34.893008 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180f6076-5909-4516-b2de-96e5e81ff4a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:34.893465 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180f6076-5909-4516-b2de-96e5e81ff4a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:34.893743 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180f6076-5909-4516-b2de-96e5e81ff4a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:34.894657 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180f6076-5909-4516-b2de-96e5e81ff4a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:42:34.895613 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180f6076-5909-4516-b2de-96e5e81ff4a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:34.900516 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180f6076-5909-4516-b2de-96e5e81ff4a2 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:34.914441 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1103/2190] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 24 06:42:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:42:34.915666 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180f6076-5909-4516-b2de-96e5e81ff4a2 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:34.931619 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180f6076-5909-4516-b2de-96e5e81ff4a2 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:34.939026 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff66e8ea-d069-4c05-9a32-907b0312b0e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:34.940469 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff66e8ea-d069-4c05-9a32-907b0312b0e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:34.940834 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff66e8ea-d069-4c05-9a32-907b0312b0e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:34.952374 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86cccab-ae5c-4ea5-98ff-bb4036a2fa25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:34.952374 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86cccab-ae5c-4ea5-98ff-bb4036a2fa25 service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:34.952623 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff66e8ea-d069-4c05-9a32-907b0312b0e4 admin admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:34.952623 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff66e8ea-d069-4c05-9a32-907b0312b0e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:34.957560 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff66e8ea-d069-4c05-9a32-907b0312b0e4 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:34.975379 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1088/2191] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:42:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:42:34.978023 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86cccab-ae5c-4ea5-98ff-bb4036a2fa25 service neutron] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:34.998004 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86cccab-ae5c-4ea5-98ff-bb4036a2fa25 service neutron] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:35.010033 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f153c78-a5a1-47a1-8abc-5b1d24d17bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:35.010979 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f153c78-a5a1-47a1-8abc-5b1d24d17bca admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:35.011611 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f153c78-a5a1-47a1-8abc-5b1d24d17bca admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:35.012172 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f153c78-a5a1-47a1-8abc-5b1d24d17bca admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:35.013028 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f153c78-a5a1-47a1-8abc-5b1d24d17bca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:35.015481 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f153c78-a5a1-47a1-8abc-5b1d24d17bca admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:35.107914 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1104/2192] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:42:34 2022] POST /identity/v3/users => generated 299 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:42:35.108982 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff66e8ea-d069-4c05-9a32-907b0312b0e4 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:35.119572 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff66e8ea-d069-4c05-9a32-907b0312b0e4 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:35.124290 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9710a97-3e66-4b0c-b32f-554542640d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:35.125075 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9710a97-3e66-4b0c-b32f-554542640d77 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:35.125404 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9710a97-3e66-4b0c-b32f-554542640d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:35.125689 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9710a97-3e66-4b0c-b32f-554542640d77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:35.126333 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9710a97-3e66-4b0c-b32f-554542640d77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:35.127990 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9710a97-3e66-4b0c-b32f-554542640d77 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:35.141587 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1105/2193] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:42:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:42:35.145829 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9710a97-3e66-4b0c-b32f-554542640d77 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:35.156955 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9710a97-3e66-4b0c-b32f-554542640d77 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:35.161767 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0588e10c-1918-44ea-bcc2-b1eb379c474e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:35.162374 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0588e10c-1918-44ea-bcc2-b1eb379c474e admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:35.162654 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0588e10c-1918-44ea-bcc2-b1eb379c474e admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:35.162972 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0588e10c-1918-44ea-bcc2-b1eb379c474e admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:35.164377 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0588e10c-1918-44ea-bcc2-b1eb379c474e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:35.166709 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0588e10c-1918-44ea-bcc2-b1eb379c474e admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:35.177438 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1106/2194] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:42:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:42:35.182002 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0588e10c-1918-44ea-bcc2-b1eb379c474e admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:35.192891 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0588e10c-1918-44ea-bcc2-b1eb379c474e admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:35.200097 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1089/2195] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:42:34 2022] POST /identity/v3/projects => generated 371 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:42:35.200936 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f153c78-a5a1-47a1-8abc-5b1d24d17bca admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:35.201066 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd5b0e1-0e7e-4e76-b41f-6034b826a276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:35.201680 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd5b0e1-0e7e-4e76-b41f-6034b826a276 admin admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:35.201962 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd5b0e1-0e7e-4e76-b41f-6034b826a276 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:35.202301 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd5b0e1-0e7e-4e76-b41f-6034b826a276 admin admin] PATH_INFO: `/v3/projects/243b82313b6b48018d38844a864d40e0/users/74d078d26bdb4ed3964c19209ef61616/roles/e1f9b167c51c4a388cbc9594db8a465d` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:35.220558 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd5b0e1-0e7e-4e76-b41f-6034b826a276 admin admin] RBAC: Authorizing `identity:create_grant(project_id=243b82313b6b48018d38844a864d40e0, user_id=74d078d26bdb4ed3964c19209ef61616, role_id=e1f9b167c51c4a388cbc9594db8a465d)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:35.223768 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd5b0e1-0e7e-4e76-b41f-6034b826a276 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:35.327585 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f153c78-a5a1-47a1-8abc-5b1d24d17bca admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:35.335114 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c744df-bab2-4460-ba01-54540d936ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:35.338474 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c744df-bab2-4460-ba01-54540d936ac4 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:35.338474 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c744df-bab2-4460-ba01-54540d936ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:35.338474 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c744df-bab2-4460-ba01-54540d936ac4 admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:35.341438 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c744df-bab2-4460-ba01-54540d936ac4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:35.341438 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c744df-bab2-4460-ba01-54540d936ac4 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:35.389222 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1107/2196] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:42:35 2022] PUT /identity/v3/projects/243b82313b6b48018d38844a864d40e0/users/74d078d26bdb4ed3964c19209ef61616/roles/e1f9b167c51c4a388cbc9594db8a465d => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:42:35.401026 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8699f468-9876-4ecb-ac53-161b136ecc5d None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:35.401354 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8699f468-9876-4ecb-ac53-161b136ecc5d None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:35.401636 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8699f468-9876-4ecb-ac53-161b136ecc5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:35.433232 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-8699f468-9876-4ecb-ac53-161b136ecc5d None None] MFA Rules not processed for user `74d078d26bdb4ed3964c19209ef61616`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:42:35.433914 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-8699f468-9876-4ecb-ac53-161b136ecc5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:35.463386 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1090/2197] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:42:35 2022] POST /identity/v3/users => generated 286 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:42:35.471593 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c744df-bab2-4460-ba01-54540d936ac4 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:35.488122 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c744df-bab2-4460-ba01-54540d936ac4 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:35.525912 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1108/2198] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:42:35 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:42:35.540687 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8699f468-9876-4ecb-ac53-161b136ecc5d None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:35.542464 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-8699f468-9876-4ecb-ac53-161b136ecc5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:35.577922 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd0b32a-c863-460f-8d65-a6790a2fc21b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:35.578743 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd0b32a-c863-460f-8d65-a6790a2fc21b admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:35.579031 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd0b32a-c863-460f-8d65-a6790a2fc21b admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:35.579304 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd0b32a-c863-460f-8d65-a6790a2fc21b admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:35.579964 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd0b32a-c863-460f-8d65-a6790a2fc21b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:35.581761 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd0b32a-c863-460f-8d65-a6790a2fc21b admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:35.596545 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1091/2199] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:42:35 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:42:35.598121 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd0b32a-c863-460f-8d65-a6790a2fc21b admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:35.601384 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-9fd0b32a-c863-460f-8d65-a6790a2fc21b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:35.625618 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd0b32a-c863-460f-8d65-a6790a2fc21b admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:35.631306 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3347a9b-8deb-4683-8aa9-cb9221a6e464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:35.632017 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3347a9b-8deb-4683-8aa9-cb9221a6e464 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:35.632905 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3347a9b-8deb-4683-8aa9-cb9221a6e464 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:35.633271 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3347a9b-8deb-4683-8aa9-cb9221a6e464 admin admin] PATH_INFO: `/v3/projects/234d261de8db404f930ae38295fe6b4b` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:35.634424 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3347a9b-8deb-4683-8aa9-cb9221a6e464 admin admin] RBAC: Authorizing `identity:delete_project(project_id=234d261de8db404f930ae38295fe6b4b)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:35.636729 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3347a9b-8deb-4683-8aa9-cb9221a6e464 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:35.643450 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8699f468-9876-4ecb-ac53-161b136ecc5d None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:35.647086 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c7fb4f-b476-47e4-a327-3ef21598a66a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36639dce4f97412f89d71367f650b901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:35.647717 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c7fb4f-b476-47e4-a327-3ef21598a66a service cinder] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:35.648052 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c7fb4f-b476-47e4-a327-3ef21598a66a service cinder] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:35.648827 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c7fb4f-b476-47e4-a327-3ef21598a66a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:35.659364 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c7fb4f-b476-47e4-a327-3ef21598a66a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:35.661008 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c7fb4f-b476-47e4-a327-3ef21598a66a service cinder] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:35.672089 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1109/2200] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:42:35 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:42:35.673395 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c7fb4f-b476-47e4-a327-3ef21598a66a service cinder] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:35.685127 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c7fb4f-b476-47e4-a327-3ef21598a66a service cinder] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:35.690798 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb825836-724f-4591-9b9a-ac21f46d81cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:35.691429 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb825836-724f-4591-9b9a-ac21f46d81cb admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:35.691713 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb825836-724f-4591-9b9a-ac21f46d81cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:35.691989 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb825836-724f-4591-9b9a-ac21f46d81cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:35.692651 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb825836-724f-4591-9b9a-ac21f46d81cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:35.694510 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb825836-724f-4591-9b9a-ac21f46d81cb admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:35.714748 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1110/2201] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:42:35 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:42:35.724945 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb825836-724f-4591-9b9a-ac21f46d81cb admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:35.736471 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb825836-724f-4591-9b9a-ac21f46d81cb admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:35.744872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4374aa5-1ed8-4cc4-bc7c-ab6ac0da86d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:35.744872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4374aa5-1ed8-4cc4-bc7c-ab6ac0da86d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:35.744872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4374aa5-1ed8-4cc4-bc7c-ab6ac0da86d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:35.744872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4374aa5-1ed8-4cc4-bc7c-ab6ac0da86d8 admin admin] PATH_INFO: `/v3/projects/7e598b3fe46c482c9c0ccbdb86188674/users/59647b5e651c41eca1a62680df4369e3/roles/e1f9b167c51c4a388cbc9594db8a465d` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:35.764809 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4374aa5-1ed8-4cc4-bc7c-ab6ac0da86d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e598b3fe46c482c9c0ccbdb86188674, user_id=59647b5e651c41eca1a62680df4369e3, role_id=e1f9b167c51c4a388cbc9594db8a465d)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:35.767971 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4374aa5-1ed8-4cc4-bc7c-ab6ac0da86d8 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:35.875102 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1111/2202] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:42:35 2022] PUT /identity/v3/projects/7e598b3fe46c482c9c0ccbdb86188674/users/59647b5e651c41eca1a62680df4369e3/roles/e1f9b167c51c4a388cbc9594db8a465d => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:42:35.885718 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:42:35.888637 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2810c97d-6dbe-49ab-8ecc-b430676a53b9 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:35.889556 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2810c97d-6dbe-49ab-8ecc-b430676a53b9 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:35.889556 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2810c97d-6dbe-49ab-8ecc-b430676a53b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:35.904275 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-f3347a9b-8deb-4683-8aa9-cb9221a6e464 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '234d261de8db404f930ae38295fe6b4b', 'request_id': 'req-f3347a9b-8deb-4683-8aa9-cb9221a6e464'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:35.911766 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-f3347a9b-8deb-4683-8aa9-cb9221a6e464 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '234d261de8db404f930ae38295fe6b4b', 'request_id': 'req-f3347a9b-8deb-4683-8aa9-cb9221a6e464'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:35.943997 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-2810c97d-6dbe-49ab-8ecc-b430676a53b9 None None] MFA Rules not processed for user `59647b5e651c41eca1a62680df4369e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:42:35.946103 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-2810c97d-6dbe-49ab-8ecc-b430676a53b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:36.027619 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-f3347a9b-8deb-4683-8aa9-cb9221a6e464 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '234d261de8db404f930ae38295fe6b4b', 'request_id': 'req-f3347a9b-8deb-4683-8aa9-cb9221a6e464'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:36.040860 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-f3347a9b-8deb-4683-8aa9-cb9221a6e464 admin admin] The token cache is being invalidate because project 234d261de8db404f930ae38295fe6b4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:42:36.041074 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-f3347a9b-8deb-4683-8aa9-cb9221a6e464 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:36.041596 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-f3347a9b-8deb-4683-8aa9-cb9221a6e464 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:36.043416 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1092/2203] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:42:35 2022] DELETE /identity/v3/projects/234d261de8db404f930ae38295fe6b4b => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:42:36.046785 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1112/2204] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:42:35 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:42:36.052053 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee554a15-b852-44d2-8790-235809bfb42e None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:36.052053 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee554a15-b852-44d2-8790-235809bfb42e None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:36.052311 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee554a15-b852-44d2-8790-235809bfb42e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:36.060370 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2810c97d-6dbe-49ab-8ecc-b430676a53b9 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:36.061011 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:42:36.067797 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-2810c97d-6dbe-49ab-8ecc-b430676a53b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:36.078161 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-ee554a15-b852-44d2-8790-235809bfb42e None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:42:36.079391 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-ee554a15-b852-44d2-8790-235809bfb42e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:36.172195 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2810c97d-6dbe-49ab-8ecc-b430676a53b9 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:36.176034 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1093/2205] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:42:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:42:36.183071 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffec5a30-a2b9-476a-88ed-ee15d4759d25 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b331b6ddfe46aeae7b52fcbbc27771', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:36.183763 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffec5a30-a2b9-476a-88ed-ee15d4759d25 service glance] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:36.184053 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffec5a30-a2b9-476a-88ed-ee15d4759d25 service glance] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:36.184328 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffec5a30-a2b9-476a-88ed-ee15d4759d25 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:36.186622 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-ffec5a30-a2b9-476a-88ed-ee15d4759d25 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:36.187704 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a567fdeb-22e6-41b0-a4dc-42a6d0d745bf None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:36.188035 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a567fdeb-22e6-41b0-a4dc-42a6d0d745bf None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:36.188426 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a567fdeb-22e6-41b0-a4dc-42a6d0d745bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:36.200230 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffec5a30-a2b9-476a-88ed-ee15d4759d25 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:36.201730 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffec5a30-a2b9-476a-88ed-ee15d4759d25 service glance] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:36.211868 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-a567fdeb-22e6-41b0-a4dc-42a6d0d745bf None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:42:36.213067 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1113/2206] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:42:36 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:42:36.213710 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-a567fdeb-22e6-41b0-a4dc-42a6d0d745bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:36.223197 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1094/2207] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:42:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:42:36.228964 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffec5a30-a2b9-476a-88ed-ee15d4759d25 service glance] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:36.247128 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffec5a30-a2b9-476a-88ed-ee15d4759d25 service glance] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:36.254823 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f58098-b55b-4813-ad98-e4443dbd651d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:36.256004 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f58098-b55b-4813-ad98-e4443dbd651d admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:36.256434 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f58098-b55b-4813-ad98-e4443dbd651d admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:36.256719 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f58098-b55b-4813-ad98-e4443dbd651d admin admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:36.259145 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f58098-b55b-4813-ad98-e4443dbd651d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:42:36.259689 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f58098-b55b-4813-ad98-e4443dbd651d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:36.262712 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f58098-b55b-4813-ad98-e4443dbd651d admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:36.284351 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1114/2208] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 24 06:42:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:42:36.289513 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a567fdeb-22e6-41b0-a4dc-42a6d0d745bf None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:36.299294 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a567fdeb-22e6-41b0-a4dc-42a6d0d745bf None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:36.304074 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0a51de-ad76-4d7e-ac66-59cb7a9ec978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:36.305406 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0a51de-ad76-4d7e-ac66-59cb7a9ec978 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:36.305628 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0a51de-ad76-4d7e-ac66-59cb7a9ec978 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:36.306109 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0a51de-ad76-4d7e-ac66-59cb7a9ec978 admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:36.306753 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0a51de-ad76-4d7e-ac66-59cb7a9ec978 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:36.308667 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0a51de-ad76-4d7e-ac66-59cb7a9ec978 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:36.410763 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1095/2209] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:42:36 2022] POST /identity/v3/projects => generated 379 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:42:36.412053 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f58098-b55b-4813-ad98-e4443dbd651d admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:36.424552 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f58098-b55b-4813-ad98-e4443dbd651d admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:36.484446 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f14de7d-528b-45f1-9d5b-8a9a884e296a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:36.485088 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f14de7d-528b-45f1-9d5b-8a9a884e296a admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:36.485399 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f14de7d-528b-45f1-9d5b-8a9a884e296a admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:36.485698 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f14de7d-528b-45f1-9d5b-8a9a884e296a admin admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:36.486338 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f14de7d-528b-45f1-9d5b-8a9a884e296a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:36.488273 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f14de7d-528b-45f1-9d5b-8a9a884e296a admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:36.595060 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1115/2210] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:42:36 2022] POST /identity/v3/users => generated 290 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:42:36.599455 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0a51de-ad76-4d7e-ac66-59cb7a9ec978 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:36.613428 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0a51de-ad76-4d7e-ac66-59cb7a9ec978 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:36.619097 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0178a20-e5c1-4852-9c8a-1d8cb90e7818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:36.619834 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0178a20-e5c1-4852-9c8a-1d8cb90e7818 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:36.620182 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0178a20-e5c1-4852-9c8a-1d8cb90e7818 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:36.620572 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0178a20-e5c1-4852-9c8a-1d8cb90e7818 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:36.621314 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0178a20-e5c1-4852-9c8a-1d8cb90e7818 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:36.623163 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0178a20-e5c1-4852-9c8a-1d8cb90e7818 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:36.638869 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1096/2211] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:42:36 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:42:36.645057 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f14de7d-528b-45f1-9d5b-8a9a884e296a admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:36.661131 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f14de7d-528b-45f1-9d5b-8a9a884e296a admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:36.665923 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d108b99-db1a-42ba-90a2-e00453c2cbc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:36.666636 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d108b99-db1a-42ba-90a2-e00453c2cbc4 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:36.666922 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d108b99-db1a-42ba-90a2-e00453c2cbc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:36.667211 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d108b99-db1a-42ba-90a2-e00453c2cbc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:36.668009 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d108b99-db1a-42ba-90a2-e00453c2cbc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:36.669666 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d108b99-db1a-42ba-90a2-e00453c2cbc4 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:36.681705 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1116/2212] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:42:36 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:42:36.687809 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0178a20-e5c1-4852-9c8a-1d8cb90e7818 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:36.706960 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0178a20-e5c1-4852-9c8a-1d8cb90e7818 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:36.709253 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ba2d03-aa16-4e88-b4a9-da3d4afffbc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:36.709930 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ba2d03-aa16-4e88-b4a9-da3d4afffbc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:36.710212 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ba2d03-aa16-4e88-b4a9-da3d4afffbc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:36.711113 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ba2d03-aa16-4e88-b4a9-da3d4afffbc5 admin admin] PATH_INFO: `/v3/projects/a97277d100304d6aa689f73174562474/users/594e9324c44540568a81fd5fdb980efb/roles/e1f9b167c51c4a388cbc9594db8a465d` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:36.735967 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ba2d03-aa16-4e88-b4a9-da3d4afffbc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a97277d100304d6aa689f73174562474, user_id=594e9324c44540568a81fd5fdb980efb, role_id=e1f9b167c51c4a388cbc9594db8a465d)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:36.735967 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ba2d03-aa16-4e88-b4a9-da3d4afffbc5 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:36.847000 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1097/2213] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:42:36 2022] PUT /identity/v3/projects/a97277d100304d6aa689f73174562474/users/594e9324c44540568a81fd5fdb980efb/roles/e1f9b167c51c4a388cbc9594db8a465d => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:42:36.851145 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f995fdb-d8f6-4133-b660-a6a3e4b45136 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:36.851562 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f995fdb-d8f6-4133-b660-a6a3e4b45136 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:36.851834 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f995fdb-d8f6-4133-b660-a6a3e4b45136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:36.890773 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-1f995fdb-d8f6-4133-b660-a6a3e4b45136 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:42:36.892282 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-1f995fdb-d8f6-4133-b660-a6a3e4b45136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:36.976022 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1117/2214] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:42:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:42:36.986253 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ba2d03-aa16-4e88-b4a9-da3d4afffbc5 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:36.987995 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-15ba2d03-aa16-4e88-b4a9-da3d4afffbc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:37.047159 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ba2d03-aa16-4e88-b4a9-da3d4afffbc5 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:37.050858 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1dd29e-4f12-490a-b9f8-e3debf1a426f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:37.051493 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1dd29e-4f12-490a-b9f8-e3debf1a426f service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:37.051779 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1dd29e-4f12-490a-b9f8-e3debf1a426f service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:37.052063 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1dd29e-4f12-490a-b9f8-e3debf1a426f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:37.062121 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1dd29e-4f12-490a-b9f8-e3debf1a426f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:37.069638 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1dd29e-4f12-490a-b9f8-e3debf1a426f service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:37.074018 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1098/2215] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:42:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:42:40.067329 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:42:40.531582 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:42:41.828012 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:42:42.090520 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:42:43.427792 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:42:43.661144 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:42:44.119789 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3300e42-a4de-4990-adc9-6654732007d7 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:44.119940 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3300e42-a4de-4990-adc9-6654732007d7 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:44.120226 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3300e42-a4de-4990-adc9-6654732007d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:44.173745 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-e3300e42-a4de-4990-adc9-6654732007d7 None None] MFA Rules not processed for user `594e9324c44540568a81fd5fdb980efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:42:44.175111 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-e3300e42-a4de-4990-adc9-6654732007d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:44.336709 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1118/2216] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:42:44 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:42:44.338907 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1dd29e-4f12-490a-b9f8-e3debf1a426f service neutron] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:44.353374 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1dd29e-4f12-490a-b9f8-e3debf1a426f service neutron] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:44.360389 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bbf8cd-56ab-4999-9c6e-b671fc1283eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:44.361288 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bbf8cd-56ab-4999-9c6e-b671fc1283eb admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:44.361739 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bbf8cd-56ab-4999-9c6e-b671fc1283eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:44.362174 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bbf8cd-56ab-4999-9c6e-b671fc1283eb admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:44.362963 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bbf8cd-56ab-4999-9c6e-b671fc1283eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:44.365157 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bbf8cd-56ab-4999-9c6e-b671fc1283eb admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:44.471728 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1099/2217] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:42:44 2022] POST /identity/v3/projects => generated 377 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:42:44.473229 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3300e42-a4de-4990-adc9-6654732007d7 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:44.494430 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3300e42-a4de-4990-adc9-6654732007d7 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:44.565961 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631513f0-92c6-40fc-b3d3-497d0004f692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:44.566591 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631513f0-92c6-40fc-b3d3-497d0004f692 admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:44.566871 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631513f0-92c6-40fc-b3d3-497d0004f692 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:44.567145 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631513f0-92c6-40fc-b3d3-497d0004f692 admin admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:44.567778 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631513f0-92c6-40fc-b3d3-497d0004f692 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:44.570247 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631513f0-92c6-40fc-b3d3-497d0004f692 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:44.599226 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:42:44.690813 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:42:44.701311 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1119/2218] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:42:44 2022] POST /identity/v3/users => generated 295 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:42:44.703844 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bbf8cd-56ab-4999-9c6e-b671fc1283eb admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:44.716217 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bbf8cd-56ab-4999-9c6e-b671fc1283eb admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:44.731206 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33635fc-d673-42da-b380-a0100eac3106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:44.732284 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33635fc-d673-42da-b380-a0100eac3106 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:44.733273 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33635fc-d673-42da-b380-a0100eac3106 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:44.733273 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33635fc-d673-42da-b380-a0100eac3106 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:44.734184 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33635fc-d673-42da-b380-a0100eac3106 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:44.743119 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33635fc-d673-42da-b380-a0100eac3106 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:44.768818 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1100/2219] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:42:44 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:42:44.773227 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631513f0-92c6-40fc-b3d3-497d0004f692 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:44.784406 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631513f0-92c6-40fc-b3d3-497d0004f692 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:44.789682 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ded492-c8bb-425c-816a-0a8443481790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:44.790401 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ded492-c8bb-425c-816a-0a8443481790 admin admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:44.790684 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ded492-c8bb-425c-816a-0a8443481790 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:44.790964 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ded492-c8bb-425c-816a-0a8443481790 admin admin] PATH_INFO: `/v3/projects/6148a6eba0e64d2bbec03dcec1b225e9/users/d138aba8c4654fd0a3e08ea068315492/roles/486144ad1bcd417e86acdbc1d14327bf` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:44.808160 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ded492-c8bb-425c-816a-0a8443481790 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6148a6eba0e64d2bbec03dcec1b225e9, user_id=d138aba8c4654fd0a3e08ea068315492, role_id=486144ad1bcd417e86acdbc1d14327bf)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:44.810913 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ded492-c8bb-425c-816a-0a8443481790 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:44.879676 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1120/2220] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:42:44 2022] PUT /identity/v3/projects/6148a6eba0e64d2bbec03dcec1b225e9/users/d138aba8c4654fd0a3e08ea068315492/roles/486144ad1bcd417e86acdbc1d14327bf => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:42:44.886115 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33635fc-d673-42da-b380-a0100eac3106 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:44.896608 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33635fc-d673-42da-b380-a0100eac3106 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:44.955850 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a9ef52-984e-48b6-b9b5-40a51c51992d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:44.956396 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a9ef52-984e-48b6-b9b5-40a51c51992d admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:44.956740 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a9ef52-984e-48b6-b9b5-40a51c51992d admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:44.957031 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a9ef52-984e-48b6-b9b5-40a51c51992d admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:44.957635 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a9ef52-984e-48b6-b9b5-40a51c51992d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:44.959237 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a9ef52-984e-48b6-b9b5-40a51c51992d admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:44.971036 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1101/2221] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:42:44 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:42:44.975367 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ded492-c8bb-425c-816a-0a8443481790 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:44.987437 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ded492-c8bb-425c-816a-0a8443481790 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:44.995613 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab4a29e-9a15-459a-b59c-565428198c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:44.996297 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab4a29e-9a15-459a-b59c-565428198c88 admin admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:44.996585 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab4a29e-9a15-459a-b59c-565428198c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:44.996893 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab4a29e-9a15-459a-b59c-565428198c88 admin admin] PATH_INFO: `/v3/projects/6148a6eba0e64d2bbec03dcec1b225e9/users/d138aba8c4654fd0a3e08ea068315492/roles/486144ad1bcd417e86acdbc1d14327bf` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:44.998909 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab4a29e-9a15-459a-b59c-565428198c88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6148a6eba0e64d2bbec03dcec1b225e9, user_id=d138aba8c4654fd0a3e08ea068315492, role_id=486144ad1bcd417e86acdbc1d14327bf)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:45.001953 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab4a29e-9a15-459a-b59c-565428198c88 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:45.014376 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1121/2222] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:42:44 2022] PUT /identity/v3/projects/6148a6eba0e64d2bbec03dcec1b225e9/users/d138aba8c4654fd0a3e08ea068315492/roles/486144ad1bcd417e86acdbc1d14327bf => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:42:46.547548 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:42:46.646627 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:42:48.261620 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:42:48.562915 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a9ef52-984e-48b6-b9b5-40a51c51992d admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:48.564268 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-b3a9ef52-984e-48b6-b9b5-40a51c51992d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:48.587163 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:42:48.593384 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a9ef52-984e-48b6-b9b5-40a51c51992d admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:48.774314 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d3b9c6-16fa-4b74-8ebd-bdf4adc2172d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:48.774978 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d3b9c6-16fa-4b74-8ebd-bdf4adc2172d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:48.775538 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d3b9c6-16fa-4b74-8ebd-bdf4adc2172d admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:48.775888 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d3b9c6-16fa-4b74-8ebd-bdf4adc2172d admin admin] PATH_INFO: `/v3/users/74d078d26bdb4ed3964c19209ef61616` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:48.779700 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d3b9c6-16fa-4b74-8ebd-bdf4adc2172d admin admin] RBAC: Authorizing `identity:delete_user(user_id=74d078d26bdb4ed3964c19209ef61616)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:48.785210 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d3b9c6-16fa-4b74-8ebd-bdf4adc2172d admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:49.083659 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-b7d3b9c6-16fa-4b74-8ebd-bdf4adc2172d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74d078d26bdb4ed3964c19209ef61616'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:49.084550 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-b7d3b9c6-16fa-4b74-8ebd-bdf4adc2172d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74d078d26bdb4ed3964c19209ef61616'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:49.088925 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-b7d3b9c6-16fa-4b74-8ebd-bdf4adc2172d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74d078d26bdb4ed3964c19209ef61616'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:49.109109 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-b7d3b9c6-16fa-4b74-8ebd-bdf4adc2172d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74d078d26bdb4ed3964c19209ef61616'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:49.250646 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-b7d3b9c6-16fa-4b74-8ebd-bdf4adc2172d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74d078d26bdb4ed3964c19209ef61616'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:49.309598 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1102/2223] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:42:48 2022] DELETE /identity/v3/users/74d078d26bdb4ed3964c19209ef61616 => generated 0 bytes in 749 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:42:49.314544 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066ec792-fc29-48d4-97af-372d397a0422 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:49.315086 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066ec792-fc29-48d4-97af-372d397a0422 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:49.315642 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066ec792-fc29-48d4-97af-372d397a0422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:49.355726 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-066ec792-fc29-48d4-97af-372d397a0422 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:42:49.357232 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-066ec792-fc29-48d4-97af-372d397a0422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:49.544964 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1122/2224] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:42:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:42:49.563567 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d3b9c6-16fa-4b74-8ebd-bdf4adc2172d admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:49.566026 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-b7d3b9c6-16fa-4b74-8ebd-bdf4adc2172d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:49.677765 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d3b9c6-16fa-4b74-8ebd-bdf4adc2172d admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:49.682236 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5211aa50-05cb-4d14-b847-bfcc8313117f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:49.683647 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5211aa50-05cb-4d14-b847-bfcc8313117f service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:49.684079 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5211aa50-05cb-4d14-b847-bfcc8313117f service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:49.685293 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5211aa50-05cb-4d14-b847-bfcc8313117f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:49.706671 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5211aa50-05cb-4d14-b847-bfcc8313117f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:49.709316 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5211aa50-05cb-4d14-b847-bfcc8313117f service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:49.722281 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1103/2225] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:42:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:42:49.854020 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:42:50.106650 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:42:50.318841 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:42:50.656366 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066ec792-fc29-48d4-97af-372d397a0422 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:50.662462 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-066ec792-fc29-48d4-97af-372d397a0422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:50.700278 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066ec792-fc29-48d4-97af-372d397a0422 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:50.730111 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cb5ea8-b4e6-4e71-801a-0aba4125a2fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:50.730853 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cb5ea8-b4e6-4e71-801a-0aba4125a2fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:50.731368 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cb5ea8-b4e6-4e71-801a-0aba4125a2fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:50.731804 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cb5ea8-b4e6-4e71-801a-0aba4125a2fa admin admin] PATH_INFO: `/v3/projects/243b82313b6b48018d38844a864d40e0` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:50.734029 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cb5ea8-b4e6-4e71-801a-0aba4125a2fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=243b82313b6b48018d38844a864d40e0)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:50.744038 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cb5ea8-b4e6-4e71-801a-0aba4125a2fa admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:50.926631 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-e2cb5ea8-b4e6-4e71-801a-0aba4125a2fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '243b82313b6b48018d38844a864d40e0', 'request_id': 'req-e2cb5ea8-b4e6-4e71-801a-0aba4125a2fa'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:51.026954 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-e2cb5ea8-b4e6-4e71-801a-0aba4125a2fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '243b82313b6b48018d38844a864d40e0', 'request_id': 'req-e2cb5ea8-b4e6-4e71-801a-0aba4125a2fa'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:51.038886 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-e2cb5ea8-b4e6-4e71-801a-0aba4125a2fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '243b82313b6b48018d38844a864d40e0', 'request_id': 'req-e2cb5ea8-b4e6-4e71-801a-0aba4125a2fa'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:51.064858 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-e2cb5ea8-b4e6-4e71-801a-0aba4125a2fa admin admin] The token cache is being invalidate because project 243b82313b6b48018d38844a864d40e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81723) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:42:51.066036 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-e2cb5ea8-b4e6-4e71-801a-0aba4125a2fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:51.066958 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-e2cb5ea8-b4e6-4e71-801a-0aba4125a2fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:42:51.070234 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1123/2226] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:42:50 2022] DELETE /identity/v3/projects/243b82313b6b48018d38844a864d40e0 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:42:51.077026 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a7186f-3e45-4c1c-9353-e81faf126d01 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:51.082801 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a7186f-3e45-4c1c-9353-e81faf126d01 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:51.082801 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a7186f-3e45-4c1c-9353-e81faf126d01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:51.112231 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-a6a7186f-3e45-4c1c-9353-e81faf126d01 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:42:51.112841 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:42:51.115345 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-a6a7186f-3e45-4c1c-9353-e81faf126d01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:51.193239 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1104/2227] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:42:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:42:51.203952 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd884b0-808e-4e69-8e3f-1b860ced2007 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:51.204286 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd884b0-808e-4e69-8e3f-1b860ced2007 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:51.204565 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd884b0-808e-4e69-8e3f-1b860ced2007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:51.229519 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-8bd884b0-808e-4e69-8e3f-1b860ced2007 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:42:51.230772 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-8bd884b0-808e-4e69-8e3f-1b860ced2007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:51.242821 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1124/2228] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:42:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:42:51.245722 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a7186f-3e45-4c1c-9353-e81faf126d01 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:51.256111 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a7186f-3e45-4c1c-9353-e81faf126d01 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:51.262274 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9005560-1f30-4847-9cec-633db172f750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:51.263137 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9005560-1f30-4847-9cec-633db172f750 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:51.263550 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9005560-1f30-4847-9cec-633db172f750 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:51.263891 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9005560-1f30-4847-9cec-633db172f750 admin admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:51.264679 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9005560-1f30-4847-9cec-633db172f750 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:42:51.265265 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9005560-1f30-4847-9cec-633db172f750 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:51.267129 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9005560-1f30-4847-9cec-633db172f750 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:51.279945 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1105/2229] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 24 06:42:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:42:51.294792 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd884b0-808e-4e69-8e3f-1b860ced2007 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:51.305529 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd884b0-808e-4e69-8e3f-1b860ced2007 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:51.322398 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d49593-0090-4f48-af22-5b96e339a87b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:51.323134 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d49593-0090-4f48-af22-5b96e339a87b admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:51.323409 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d49593-0090-4f48-af22-5b96e339a87b admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:51.323676 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d49593-0090-4f48-af22-5b96e339a87b admin admin] PATH_INFO: `/v3/projects` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:51.324566 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d49593-0090-4f48-af22-5b96e339a87b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:51.328412 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d49593-0090-4f48-af22-5b96e339a87b admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:51.431248 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9005560-1f30-4847-9cec-633db172f750 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:51.436062 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:42:51.438605 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1125/2230] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:42:51 2022] POST /identity/v3/projects => generated 381 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:42:51.445041 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9005560-1f30-4847-9cec-633db172f750 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:51.518634 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457c67ef-3f77-4404-9f3c-0119368130dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:51.519272 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457c67ef-3f77-4404-9f3c-0119368130dd admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:51.519547 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457c67ef-3f77-4404-9f3c-0119368130dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:51.520072 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457c67ef-3f77-4404-9f3c-0119368130dd admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:51.520732 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457c67ef-3f77-4404-9f3c-0119368130dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:51.522914 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457c67ef-3f77-4404-9f3c-0119368130dd admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:51.547072 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:42:51.618425 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1106/2231] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:42:51 2022] POST /identity/v3/users => generated 291 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:42:51.621048 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d49593-0090-4f48-af22-5b96e339a87b admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:51.640666 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d49593-0090-4f48-af22-5b96e339a87b admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:51.647367 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b250d9-9fb8-43e9-993a-735e1990c851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:51.649802 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b250d9-9fb8-43e9-993a-735e1990c851 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:51.649802 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b250d9-9fb8-43e9-993a-735e1990c851 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:51.649802 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b250d9-9fb8-43e9-993a-735e1990c851 admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:51.652630 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b250d9-9fb8-43e9-993a-735e1990c851 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:51.658511 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b250d9-9fb8-43e9-993a-735e1990c851 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:51.682888 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1126/2232] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:42:51 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:42:51.687762 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457c67ef-3f77-4404-9f3c-0119368130dd admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:51.715078 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457c67ef-3f77-4404-9f3c-0119368130dd admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:51.730147 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75581c6c-0aee-495e-8907-474f9f485f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:51.730147 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75581c6c-0aee-495e-8907-474f9f485f55 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:51.730147 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75581c6c-0aee-495e-8907-474f9f485f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:51.730147 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75581c6c-0aee-495e-8907-474f9f485f55 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:51.730604 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75581c6c-0aee-495e-8907-474f9f485f55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:51.730604 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75581c6c-0aee-495e-8907-474f9f485f55 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:51.796603 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1107/2233] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:42:51 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:42:51.801062 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b250d9-9fb8-43e9-993a-735e1990c851 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:51.822641 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b250d9-9fb8-43e9-993a-735e1990c851 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:51.837104 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50ee5ff-0cf9-450c-bd43-f9227ff3703e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:51.845459 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50ee5ff-0cf9-450c-bd43-f9227ff3703e admin admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:51.845459 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50ee5ff-0cf9-450c-bd43-f9227ff3703e admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:51.845459 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50ee5ff-0cf9-450c-bd43-f9227ff3703e admin admin] PATH_INFO: `/v3/projects/e36bdfb532924d8ebe001112e285fcdc/users/fc1fefc915e941628a01336564368f81/roles/e1f9b167c51c4a388cbc9594db8a465d` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:51.875727 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50ee5ff-0cf9-450c-bd43-f9227ff3703e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e36bdfb532924d8ebe001112e285fcdc, user_id=fc1fefc915e941628a01336564368f81, role_id=e1f9b167c51c4a388cbc9594db8a465d)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:51.882286 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50ee5ff-0cf9-450c-bd43-f9227ff3703e admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:51.996174 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1127/2234] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:42:51 2022] PUT /identity/v3/projects/e36bdfb532924d8ebe001112e285fcdc/users/fc1fefc915e941628a01336564368f81/roles/e1f9b167c51c4a388cbc9594db8a465d => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:42:52.005716 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8872c4ae-b096-411f-aff0-0792b1e08d86 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:52.006003 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8872c4ae-b096-411f-aff0-0792b1e08d86 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:52.006476 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8872c4ae-b096-411f-aff0-0792b1e08d86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:52.032386 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-8872c4ae-b096-411f-aff0-0792b1e08d86 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:42:52.033715 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-8872c4ae-b096-411f-aff0-0792b1e08d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:52.104256 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1108/2235] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:42:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:42:52.114620 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50ee5ff-0cf9-450c-bd43-f9227ff3703e admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:52.117805 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-f50ee5ff-0cf9-450c-bd43-f9227ff3703e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:52.219128 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:42:52.233713 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50ee5ff-0cf9-450c-bd43-f9227ff3703e admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:52.238134 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046e869d-6bf8-4e89-851d-ba992d35b7aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:52.239167 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046e869d-6bf8-4e89-851d-ba992d35b7aa service neutron] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:52.239562 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046e869d-6bf8-4e89-851d-ba992d35b7aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:52.239999 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046e869d-6bf8-4e89-851d-ba992d35b7aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:52.247913 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:42:52.263049 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046e869d-6bf8-4e89-851d-ba992d35b7aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:52.269163 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046e869d-6bf8-4e89-851d-ba992d35b7aa service neutron] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:52.282032 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1128/2236] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:42:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:42:53.066781 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c5c101-ef3f-4d41-a9b9-ba31220d1319 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:53.067232 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c5c101-ef3f-4d41-a9b9-ba31220d1319 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:53.067342 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c5c101-ef3f-4d41-a9b9-ba31220d1319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:53.106207 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-02c5c101-ef3f-4d41-a9b9-ba31220d1319 None None] MFA Rules not processed for user `bb4a45146170497098d2071618ea6469`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:42:53.107746 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-02c5c101-ef3f-4d41-a9b9-ba31220d1319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:53.236878 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1109/2237] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:42:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:42:53.271269 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:42:53.438732 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:42:53.785705 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76171b1-ef9c-494a-be7f-bb636d720eda None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:53.785992 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76171b1-ef9c-494a-be7f-bb636d720eda None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:53.786277 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76171b1-ef9c-494a-be7f-bb636d720eda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:53.827259 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-f76171b1-ef9c-494a-be7f-bb636d720eda None None] MFA Rules not processed for user `d138aba8c4654fd0a3e08ea068315492`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:42:53.829102 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-f76171b1-ef9c-494a-be7f-bb636d720eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:53.926859 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1129/2238] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:42:53 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:42:53.943143 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c5c101-ef3f-4d41-a9b9-ba31220d1319 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:42:53.945807 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-02c5c101-ef3f-4d41-a9b9-ba31220d1319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:54.046708 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c5c101-ef3f-4d41-a9b9-ba31220d1319 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:42:54.053801 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1f1157-a41c-4523-9490-85b75d1fe8b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4f49c1b08c460f8ba6c451c9c92c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:42:54.055063 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1f1157-a41c-4523-9490-85b75d1fe8b8 service nova] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:42:54.055345 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1f1157-a41c-4523-9490-85b75d1fe8b8 service nova] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:42:54.055619 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1f1157-a41c-4523-9490-85b75d1fe8b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:42:54.057479 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-6b1f1157-a41c-4523-9490-85b75d1fe8b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:42:54.142108 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1f1157-a41c-4523-9490-85b75d1fe8b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:42:54.143972 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1f1157-a41c-4523-9490-85b75d1fe8b8 service nova] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:42:54.157270 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1110/2239] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:42:53 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:42:55.901060 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:42:55.999995 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:42:57.489995 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:42:57.805202 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:42:59.294845 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:42:59.661017 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:00.840207 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:01.261173 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:01.262931 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6047dcde-ef4e-4f22-8a2e-63eb368447db None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:01.263407 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6047dcde-ef4e-4f22-8a2e-63eb368447db None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:01.263525 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6047dcde-ef4e-4f22-8a2e-63eb368447db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:01.301852 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-6047dcde-ef4e-4f22-8a2e-63eb368447db None None] MFA Rules not processed for user `fc1fefc915e941628a01336564368f81`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:01.303441 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-6047dcde-ef4e-4f22-8a2e-63eb368447db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:01.386712 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1130/2240] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:43:01 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:01.391537 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1f1157-a41c-4523-9490-85b75d1fe8b8 service nova] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:01.402933 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1f1157-a41c-4523-9490-85b75d1fe8b8 service nova] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:01.408113 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bd1935-20cb-4484-a3e7-82b352769e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:01.408719 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bd1935-20cb-4484-a3e7-82b352769e5d admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:01.409004 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bd1935-20cb-4484-a3e7-82b352769e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:01.409313 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bd1935-20cb-4484-a3e7-82b352769e5d admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:01.409951 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bd1935-20cb-4484-a3e7-82b352769e5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:01.411848 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bd1935-20cb-4484-a3e7-82b352769e5d admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:01.540497 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=91,keystone:INSERT=6,keystone:DELETE=3 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:01.547321 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1111/2241] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:43:01 2022] POST /identity/v3/projects => generated 383 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:43:01.551544 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6047dcde-ef4e-4f22-8a2e-63eb368447db None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:01.563516 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6047dcde-ef4e-4f22-8a2e-63eb368447db None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:01.637871 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2496efd4-0c55-4dd7-9b33-ee326c7189e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:01.639593 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2496efd4-0c55-4dd7-9b33-ee326c7189e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:01.639593 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2496efd4-0c55-4dd7-9b33-ee326c7189e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:01.639912 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2496efd4-0c55-4dd7-9b33-ee326c7189e2 admin admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:01.640780 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2496efd4-0c55-4dd7-9b33-ee326c7189e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:01.644105 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2496efd4-0c55-4dd7-9b33-ee326c7189e2 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:01.726153 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1131/2242] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:43:01 2022] POST /identity/v3/users => generated 298 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:43:01.730362 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bd1935-20cb-4484-a3e7-82b352769e5d admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:01.743327 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bd1935-20cb-4484-a3e7-82b352769e5d admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:01.748876 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc5cad4-e333-4d29-ae22-64fe5ff9509e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:01.749667 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc5cad4-e333-4d29-ae22-64fe5ff9509e admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:01.750203 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc5cad4-e333-4d29-ae22-64fe5ff9509e admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:01.754213 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc5cad4-e333-4d29-ae22-64fe5ff9509e admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:01.754213 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc5cad4-e333-4d29-ae22-64fe5ff9509e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:01.754213 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc5cad4-e333-4d29-ae22-64fe5ff9509e admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:01.764737 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1112/2243] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:43:01 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:43:01.769734 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2496efd4-0c55-4dd7-9b33-ee326c7189e2 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:01.781492 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2496efd4-0c55-4dd7-9b33-ee326c7189e2 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:01.787600 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e33852-ba3f-4a65-b287-19d84d47a111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:01.788900 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e33852-ba3f-4a65-b287-19d84d47a111 admin admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:01.789175 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e33852-ba3f-4a65-b287-19d84d47a111 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:01.789484 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e33852-ba3f-4a65-b287-19d84d47a111 admin admin] PATH_INFO: `/v3/projects/b1db1077e83d4ca89034cd358ef15644/users/61973fd08a0a418ba32dff8ead839d7a/roles/486144ad1bcd417e86acdbc1d14327bf` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:01.807939 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e33852-ba3f-4a65-b287-19d84d47a111 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1db1077e83d4ca89034cd358ef15644, user_id=61973fd08a0a418ba32dff8ead839d7a, role_id=486144ad1bcd417e86acdbc1d14327bf)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:01.811280 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e33852-ba3f-4a65-b287-19d84d47a111 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:01.941481 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1132/2244] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:43:01 2022] PUT /identity/v3/projects/b1db1077e83d4ca89034cd358ef15644/users/61973fd08a0a418ba32dff8ead839d7a/roles/486144ad1bcd417e86acdbc1d14327bf => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:43:01.947759 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc5cad4-e333-4d29-ae22-64fe5ff9509e admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:01.960585 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc5cad4-e333-4d29-ae22-64fe5ff9509e admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:02.026396 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d454c7f-bb7f-4c15-a67e-c1ebc2799a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:02.026979 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d454c7f-bb7f-4c15-a67e-c1ebc2799a0c admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:02.027295 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d454c7f-bb7f-4c15-a67e-c1ebc2799a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:02.027606 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d454c7f-bb7f-4c15-a67e-c1ebc2799a0c admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:02.028316 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d454c7f-bb7f-4c15-a67e-c1ebc2799a0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:02.030004 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d454c7f-bb7f-4c15-a67e-c1ebc2799a0c admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:02.042044 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1113/2245] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:43:01 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:43:02.047179 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e33852-ba3f-4a65-b287-19d84d47a111 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:02.059194 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e33852-ba3f-4a65-b287-19d84d47a111 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:02.064527 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65bff1e-ea61-4bb6-ae6f-784a94954802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:02.065235 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65bff1e-ea61-4bb6-ae6f-784a94954802 admin admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:02.065520 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65bff1e-ea61-4bb6-ae6f-784a94954802 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:02.065795 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65bff1e-ea61-4bb6-ae6f-784a94954802 admin admin] PATH_INFO: `/v3/projects/b1db1077e83d4ca89034cd358ef15644/users/61973fd08a0a418ba32dff8ead839d7a/roles/486144ad1bcd417e86acdbc1d14327bf` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:02.068996 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65bff1e-ea61-4bb6-ae6f-784a94954802 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1db1077e83d4ca89034cd358ef15644, user_id=61973fd08a0a418ba32dff8ead839d7a, role_id=486144ad1bcd417e86acdbc1d14327bf)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:02.077447 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65bff1e-ea61-4bb6-ae6f-784a94954802 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:02.094814 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1133/2246] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:43:02 2022] PUT /identity/v3/projects/b1db1077e83d4ca89034cd358ef15644/users/61973fd08a0a418ba32dff8ead839d7a/roles/486144ad1bcd417e86acdbc1d14327bf => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:43:02.381138 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=83,keystone:DELETE=12,keystone:INSERT=5 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:02.735674 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:03.551451 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:03.855746 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:05.147736 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:06.164081 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:07.689353 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:08.735021 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:09.604333 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:10.038475 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:10.350903 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f36e1b1-dd25-435c-b6b2-5da7fb6daa5f None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:10.351620 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f36e1b1-dd25-435c-b6b2-5da7fb6daa5f None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:10.352189 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f36e1b1-dd25-435c-b6b2-5da7fb6daa5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:10.388771 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-3f36e1b1-dd25-435c-b6b2-5da7fb6daa5f None None] MFA Rules not processed for user `61973fd08a0a418ba32dff8ead839d7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:10.390591 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-3f36e1b1-dd25-435c-b6b2-5da7fb6daa5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:10.475067 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1114/2247] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:43:10 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:10.488179 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65bff1e-ea61-4bb6-ae6f-784a94954802 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:10.491150 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-d65bff1e-ea61-4bb6-ae6f-784a94954802 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:10.591260 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65bff1e-ea61-4bb6-ae6f-784a94954802 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:10.595456 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71ff56c-fb22-41eb-a8f0-f6f6c308d4a5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b331b6ddfe46aeae7b52fcbbc27771', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:10.596069 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71ff56c-fb22-41eb-a8f0-f6f6c308d4a5 service glance] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:10.596370 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71ff56c-fb22-41eb-a8f0-f6f6c308d4a5 service glance] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:10.596626 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71ff56c-fb22-41eb-a8f0-f6f6c308d4a5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:10.660039 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71ff56c-fb22-41eb-a8f0-f6f6c308d4a5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:10.661559 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71ff56c-fb22-41eb-a8f0-f6f6c308d4a5 service glance] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:10.672154 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1134/2248] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:43:10 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:43:10.966748 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1824d020-12a0-44e9-a0db-15dad07d80ba None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:10.967123 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1824d020-12a0-44e9-a0db-15dad07d80ba None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:10.967410 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1824d020-12a0-44e9-a0db-15dad07d80ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:10.988117 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-1824d020-12a0-44e9-a0db-15dad07d80ba None None] MFA Rules not processed for user `ea70acd744994250a2905b5dd7163dfb`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:10.989371 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-1824d020-12a0-44e9-a0db-15dad07d80ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:11.057322 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1115/2249] 10.222.0.47 () {58 vars in 1009 bytes} [Wed Aug 24 06:43:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:11.088912 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:11.099706 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71ff56c-fb22-41eb-a8f0-f6f6c308d4a5 service glance] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:11.102198 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-c71ff56c-fb22-41eb-a8f0-f6f6c308d4a5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:11.175112 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71ff56c-fb22-41eb-a8f0-f6f6c308d4a5 service glance] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:11.181458 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa004c7-b10a-4893-ac1d-68acf7242747 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f1cbaf2cd6427585f298c9d58c319a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:11.182070 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa004c7-b10a-4893-ac1d-68acf7242747 service swift] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:11.182352 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa004c7-b10a-4893-ac1d-68acf7242747 service swift] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:11.182627 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa004c7-b10a-4893-ac1d-68acf7242747 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:11.191010 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa004c7-b10a-4893-ac1d-68acf7242747 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:11.192611 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa004c7-b10a-4893-ac1d-68acf7242747 service swift] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:11.201712 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1135/2250] 10.222.0.47 () {60 vars in 1384 bytes} [Wed Aug 24 06:43:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 06:43:11.303828 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:11.423399 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:11.517862 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:11.546354 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:12.112682 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:12.392630 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:12.495636 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:12.529513 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:12.810175 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:12.845813 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:12.946568 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:12.980957 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:13.163673 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:13.191860 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:13.317637 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:13.353018 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:13.516068 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:13.575078 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:13.638360 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:13.776665 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=88,keystone:INSERT=6,keystone:DELETE=6 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:13.889935 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=97,keystone:DELETE=1,keystone:INSERT=2 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:14.024063 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:23.453691 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=81 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:23.525700 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b45a074-1a68-4bff-8c52-8c33f5e149ef None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:23.525986 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b45a074-1a68-4bff-8c52-8c33f5e149ef None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:23.526261 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b45a074-1a68-4bff-8c52-8c33f5e149ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:23.546974 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-9b45a074-1a68-4bff-8c52-8c33f5e149ef None None] MFA Rules not processed for user `fc1fefc915e941628a01336564368f81`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:23.549481 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-9b45a074-1a68-4bff-8c52-8c33f5e149ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:23.556198 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1116/2251] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:43:23 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:24.170984 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=54 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:24.748321 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa004c7-b10a-4893-ac1d-68acf7242747 service swift] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:24.749806 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-9aa004c7-b10a-4893-ac1d-68acf7242747 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:24.817840 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa004c7-b10a-4893-ac1d-68acf7242747 service swift] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:24.823083 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b28ac1e-b98c-4013-9f0b-dc98669d31f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:24.823720 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b28ac1e-b98c-4013-9f0b-dc98669d31f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:24.823996 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b28ac1e-b98c-4013-9f0b-dc98669d31f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:24.824273 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b28ac1e-b98c-4013-9f0b-dc98669d31f0 admin admin] PATH_INFO: `/v3/users/59647b5e651c41eca1a62680df4369e3` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:24.825532 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b28ac1e-b98c-4013-9f0b-dc98669d31f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59647b5e651c41eca1a62680df4369e3)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:24.827477 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b28ac1e-b98c-4013-9f0b-dc98669d31f0 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:25.086881 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-1b28ac1e-b98c-4013-9f0b-dc98669d31f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59647b5e651c41eca1a62680df4369e3'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:25.087552 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-1b28ac1e-b98c-4013-9f0b-dc98669d31f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59647b5e651c41eca1a62680df4369e3'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:25.088165 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-1b28ac1e-b98c-4013-9f0b-dc98669d31f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59647b5e651c41eca1a62680df4369e3'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:25.169204 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-1b28ac1e-b98c-4013-9f0b-dc98669d31f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59647b5e651c41eca1a62680df4369e3'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:25.183053 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-1b28ac1e-b98c-4013-9f0b-dc98669d31f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59647b5e651c41eca1a62680df4369e3'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:25.196812 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1136/2252] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:43:24 2022] DELETE /identity/v3/users/59647b5e651c41eca1a62680df4369e3 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:43:25.201475 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4ee290-89d8-4a31-b7b3-5d7f28456e60 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:25.202089 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4ee290-89d8-4a31-b7b3-5d7f28456e60 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:25.202089 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4ee290-89d8-4a31-b7b3-5d7f28456e60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:25.221652 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-be4ee290-89d8-4a31-b7b3-5d7f28456e60 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:25.222877 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-be4ee290-89d8-4a31-b7b3-5d7f28456e60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:25.284985 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1117/2253] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:43:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:25.295423 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b28ac1e-b98c-4013-9f0b-dc98669d31f0 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:25.296863 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-1b28ac1e-b98c-4013-9f0b-dc98669d31f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:25.358917 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b28ac1e-b98c-4013-9f0b-dc98669d31f0 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:25.362683 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af577320-b52b-45ae-a291-34301b30f891 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:25.363295 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af577320-b52b-45ae-a291-34301b30f891 service neutron] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:25.363581 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af577320-b52b-45ae-a291-34301b30f891 service neutron] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:25.363854 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af577320-b52b-45ae-a291-34301b30f891 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:25.375242 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af577320-b52b-45ae-a291-34301b30f891 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:25.376717 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af577320-b52b-45ae-a291-34301b30f891 service neutron] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:25.388918 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1137/2254] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:43:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:43:25.934531 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4ee290-89d8-4a31-b7b3-5d7f28456e60 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:25.934531 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-be4ee290-89d8-4a31-b7b3-5d7f28456e60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:25.951929 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4ee290-89d8-4a31-b7b3-5d7f28456e60 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:25.956953 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6dbb91-8a8b-4f17-841a-486284990f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:25.957751 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6dbb91-8a8b-4f17-841a-486284990f16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:25.958128 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6dbb91-8a8b-4f17-841a-486284990f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:25.958515 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6dbb91-8a8b-4f17-841a-486284990f16 admin admin] PATH_INFO: `/v3/projects/7e598b3fe46c482c9c0ccbdb86188674` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:25.961333 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6dbb91-8a8b-4f17-841a-486284990f16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e598b3fe46c482c9c0ccbdb86188674)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:25.964167 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6dbb91-8a8b-4f17-841a-486284990f16 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:26.086233 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-ee6dbb91-8a8b-4f17-841a-486284990f16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e598b3fe46c482c9c0ccbdb86188674', 'request_id': 'req-ee6dbb91-8a8b-4f17-841a-486284990f16'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:26.092148 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-ee6dbb91-8a8b-4f17-841a-486284990f16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e598b3fe46c482c9c0ccbdb86188674', 'request_id': 'req-ee6dbb91-8a8b-4f17-841a-486284990f16'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:26.164292 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-ee6dbb91-8a8b-4f17-841a-486284990f16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e598b3fe46c482c9c0ccbdb86188674', 'request_id': 'req-ee6dbb91-8a8b-4f17-841a-486284990f16'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:26.175980 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-ee6dbb91-8a8b-4f17-841a-486284990f16 admin admin] The token cache is being invalidate because project 7e598b3fe46c482c9c0ccbdb86188674 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:43:26.176279 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-ee6dbb91-8a8b-4f17-841a-486284990f16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:26.176942 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-ee6dbb91-8a8b-4f17-841a-486284990f16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:26.179055 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1118/2255] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:43:25 2022] DELETE /identity/v3/projects/7e598b3fe46c482c9c0ccbdb86188674 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:43:26.183774 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0183bb-a059-42b8-8eda-e531fe41f74d None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:26.184260 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0183bb-a059-42b8-8eda-e531fe41f74d None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:26.184362 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0183bb-a059-42b8-8eda-e531fe41f74d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:26.220766 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-4b0183bb-a059-42b8-8eda-e531fe41f74d None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:26.222113 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-4b0183bb-a059-42b8-8eda-e531fe41f74d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:26.241535 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:26.284226 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1138/2256] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:43:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:26.292725 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6da06d-2bc5-4b6a-ac8f-b479674bdcd9 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:26.293286 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6da06d-2bc5-4b6a-ac8f-b479674bdcd9 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:26.293384 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6da06d-2bc5-4b6a-ac8f-b479674bdcd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:26.314599 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-1d6da06d-2bc5-4b6a-ac8f-b479674bdcd9 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:26.316061 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-1d6da06d-2bc5-4b6a-ac8f-b479674bdcd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:26.326763 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1119/2257] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:43:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:26.331367 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0183bb-a059-42b8-8eda-e531fe41f74d None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:26.342129 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0183bb-a059-42b8-8eda-e531fe41f74d None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:26.348366 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f1ca4e-d648-4f0a-a59d-c81155d8386a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:26.348974 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f1ca4e-d648-4f0a-a59d-c81155d8386a admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:26.349292 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f1ca4e-d648-4f0a-a59d-c81155d8386a admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:26.349638 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f1ca4e-d648-4f0a-a59d-c81155d8386a admin admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:26.350345 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f1ca4e-d648-4f0a-a59d-c81155d8386a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:43:26.350810 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f1ca4e-d648-4f0a-a59d-c81155d8386a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:26.352531 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f1ca4e-d648-4f0a-a59d-c81155d8386a admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:26.364664 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1139/2258] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 24 06:43:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:43:26.369276 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6da06d-2bc5-4b6a-ac8f-b479674bdcd9 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:26.378738 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6da06d-2bc5-4b6a-ac8f-b479674bdcd9 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:26.384451 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36e6201-4c02-4514-b6ea-19ac0539ec62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:26.389717 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36e6201-4c02-4514-b6ea-19ac0539ec62 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:26.389717 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36e6201-4c02-4514-b6ea-19ac0539ec62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:26.389717 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36e6201-4c02-4514-b6ea-19ac0539ec62 admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:26.389717 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36e6201-4c02-4514-b6ea-19ac0539ec62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:26.389717 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36e6201-4c02-4514-b6ea-19ac0539ec62 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:26.512580 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1120/2259] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:43:26 2022] POST /identity/v3/projects => generated 375 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:43:26.518520 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f1ca4e-d648-4f0a-a59d-c81155d8386a admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:26.530999 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f1ca4e-d648-4f0a-a59d-c81155d8386a admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:26.648017 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff28b847-628c-4545-839a-e1e9a8611f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:26.648664 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff28b847-628c-4545-839a-e1e9a8611f22 admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:26.654263 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff28b847-628c-4545-839a-e1e9a8611f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:26.654393 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff28b847-628c-4545-839a-e1e9a8611f22 admin admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:26.655364 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff28b847-628c-4545-839a-e1e9a8611f22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:26.658934 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff28b847-628c-4545-839a-e1e9a8611f22 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:26.869480 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1140/2260] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:43:26 2022] POST /identity/v3/users => generated 288 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:43:26.870795 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36e6201-4c02-4514-b6ea-19ac0539ec62 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:26.882492 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36e6201-4c02-4514-b6ea-19ac0539ec62 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:26.888413 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aaa894-c6df-41d4-b4bf-e58d16b696b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:26.889123 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aaa894-c6df-41d4-b4bf-e58d16b696b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:26.889429 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aaa894-c6df-41d4-b4bf-e58d16b696b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:26.898687 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aaa894-c6df-41d4-b4bf-e58d16b696b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:26.898687 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aaa894-c6df-41d4-b4bf-e58d16b696b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:26.898687 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aaa894-c6df-41d4-b4bf-e58d16b696b1 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:26.903630 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1121/2261] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:43:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:43:26.908228 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff28b847-628c-4545-839a-e1e9a8611f22 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:26.918972 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff28b847-628c-4545-839a-e1e9a8611f22 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:26.923830 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d5a38b-7ab9-4b9b-a724-ee1a2c0c8056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:26.924451 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d5a38b-7ab9-4b9b-a724-ee1a2c0c8056 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:26.924741 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d5a38b-7ab9-4b9b-a724-ee1a2c0c8056 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:26.925026 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d5a38b-7ab9-4b9b-a724-ee1a2c0c8056 admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:26.925726 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d5a38b-7ab9-4b9b-a724-ee1a2c0c8056 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:26.927385 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d5a38b-7ab9-4b9b-a724-ee1a2c0c8056 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:26.939360 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1141/2262] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:43:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:43:26.947693 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aaa894-c6df-41d4-b4bf-e58d16b696b1 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:26.963372 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aaa894-c6df-41d4-b4bf-e58d16b696b1 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:26.967676 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a965791f-4f82-4a3b-9c33-9db2ab3550bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:26.968346 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a965791f-4f82-4a3b-9c33-9db2ab3550bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:26.968631 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a965791f-4f82-4a3b-9c33-9db2ab3550bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:26.968904 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a965791f-4f82-4a3b-9c33-9db2ab3550bb admin admin] PATH_INFO: `/v3/projects/8519ae6912194be3ab89ddf8c83e9672/users/325314efd6384e8dbd59473e73595d57/roles/e1f9b167c51c4a388cbc9594db8a465d` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:26.988562 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a965791f-4f82-4a3b-9c33-9db2ab3550bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=8519ae6912194be3ab89ddf8c83e9672, user_id=325314efd6384e8dbd59473e73595d57, role_id=e1f9b167c51c4a388cbc9594db8a465d)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:26.993264 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a965791f-4f82-4a3b-9c33-9db2ab3550bb admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:27.137155 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1122/2263] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:43:26 2022] PUT /identity/v3/projects/8519ae6912194be3ab89ddf8c83e9672/users/325314efd6384e8dbd59473e73595d57/roles/e1f9b167c51c4a388cbc9594db8a465d => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:43:27.150408 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab8a37e-1b24-4bad-848f-a9e8cfcac725 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:27.150693 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab8a37e-1b24-4bad-848f-a9e8cfcac725 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:27.150972 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab8a37e-1b24-4bad-848f-a9e8cfcac725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:27.183625 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-fab8a37e-1b24-4bad-848f-a9e8cfcac725 None None] MFA Rules not processed for user `325314efd6384e8dbd59473e73595d57`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:27.184872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-fab8a37e-1b24-4bad-848f-a9e8cfcac725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:27.262861 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1142/2264] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:43:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:27.273963 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a965791f-4f82-4a3b-9c33-9db2ab3550bb admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:27.275728 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-a965791f-4f82-4a3b-9c33-9db2ab3550bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:27.291107 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:27.336806 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a965791f-4f82-4a3b-9c33-9db2ab3550bb admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:27.340767 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6956fd8f-4d7a-4cae-aa2f-2e8ac13f7f9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36639dce4f97412f89d71367f650b901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:27.341872 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6956fd8f-4d7a-4cae-aa2f-2e8ac13f7f9c service cinder] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:27.342160 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6956fd8f-4d7a-4cae-aa2f-2e8ac13f7f9c service cinder] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:27.342432 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6956fd8f-4d7a-4cae-aa2f-2e8ac13f7f9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:27.352072 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6956fd8f-4d7a-4cae-aa2f-2e8ac13f7f9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:27.353710 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6956fd8f-4d7a-4cae-aa2f-2e8ac13f7f9c service cinder] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:27.362407 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1123/2265] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:43:27 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:43:27.766391 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aa01b2-d05c-47f9-962d-e76acaa3436a None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:27.766681 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aa01b2-d05c-47f9-962d-e76acaa3436a None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:27.766983 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aa01b2-d05c-47f9-962d-e76acaa3436a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:27.795678 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-05aa01b2-d05c-47f9-962d-e76acaa3436a None None] MFA Rules not processed for user `ea70acd744994250a2905b5dd7163dfb`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:27.798770 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-05aa01b2-d05c-47f9-962d-e76acaa3436a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:27.813932 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:27.881632 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1143/2266] 10.222.0.47 () {58 vars in 1009 bytes} [Wed Aug 24 06:43:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:27.886840 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6956fd8f-4d7a-4cae-aa2f-2e8ac13f7f9c service cinder] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:27.888722 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-6956fd8f-4d7a-4cae-aa2f-2e8ac13f7f9c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:27.994634 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6956fd8f-4d7a-4cae-aa2f-2e8ac13f7f9c service cinder] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:28.000090 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399cfd23-f3b8-452f-93d8-e1da9fb5e613 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f1cbaf2cd6427585f298c9d58c319a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:28.000768 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399cfd23-f3b8-452f-93d8-e1da9fb5e613 service swift] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:28.001048 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399cfd23-f3b8-452f-93d8-e1da9fb5e613 service swift] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:28.001359 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399cfd23-f3b8-452f-93d8-e1da9fb5e613 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:28.012694 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399cfd23-f3b8-452f-93d8-e1da9fb5e613 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:28.014654 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399cfd23-f3b8-452f-93d8-e1da9fb5e613 service swift] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:28.027805 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1124/2267] 10.222.0.47 () {60 vars in 1384 bytes} [Wed Aug 24 06:43:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 06:43:34.872706 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d455d1f2-9915-43a9-b8b6-4090d83497c0 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:34.873398 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d455d1f2-9915-43a9-b8b6-4090d83497c0 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:34.873398 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d455d1f2-9915-43a9-b8b6-4090d83497c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:34.897464 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-d455d1f2-9915-43a9-b8b6-4090d83497c0 None None] MFA Rules not processed for user `ea70acd744994250a2905b5dd7163dfb`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:34.898939 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-d455d1f2-9915-43a9-b8b6-4090d83497c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:34.906726 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1144/2268] 10.222.0.47 () {58 vars in 1009 bytes} [Wed Aug 24 06:43:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:34.920698 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399cfd23-f3b8-452f-93d8-e1da9fb5e613 service swift] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:34.935881 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399cfd23-f3b8-452f-93d8-e1da9fb5e613 service swift] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:34.941633 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed06e798-23ca-4819-80d0-3f34223b4ff7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f1cbaf2cd6427585f298c9d58c319a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:34.942247 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed06e798-23ca-4819-80d0-3f34223b4ff7 service swift] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:34.942529 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed06e798-23ca-4819-80d0-3f34223b4ff7 service swift] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:34.943044 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed06e798-23ca-4819-80d0-3f34223b4ff7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:34.952729 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed06e798-23ca-4819-80d0-3f34223b4ff7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:34.956065 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed06e798-23ca-4819-80d0-3f34223b4ff7 service swift] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:34.964616 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1125/2269] 10.222.0.47 () {60 vars in 1384 bytes} [Wed Aug 24 06:43:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 06:43:37.197652 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e80b955-9a70-4ac9-ad02-99f0912ec6d7 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:37.198275 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e80b955-9a70-4ac9-ad02-99f0912ec6d7 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:37.198275 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e80b955-9a70-4ac9-ad02-99f0912ec6d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:37.240388 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-1e80b955-9a70-4ac9-ad02-99f0912ec6d7 None None] MFA Rules not processed for user `594e9324c44540568a81fd5fdb980efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:37.242908 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-1e80b955-9a70-4ac9-ad02-99f0912ec6d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:37.352727 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1145/2270] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:43:37 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:44.518925 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed06e798-23ca-4819-80d0-3f34223b4ff7 service swift] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:44.528844 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-ed06e798-23ca-4819-80d0-3f34223b4ff7 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:44.683106 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed06e798-23ca-4819-80d0-3f34223b4ff7 service swift] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:44.702876 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b4d0f8-71b2-4a48-9fe5-ed070e2abbd0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '306188d42dbd4b5b84082ab3842eaf9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:44.703523 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b4d0f8-71b2-4a48-9fe5-ed070e2abbd0 service placement] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:44.703808 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b4d0f8-71b2-4a48-9fe5-ed070e2abbd0 service placement] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:44.704085 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b4d0f8-71b2-4a48-9fe5-ed070e2abbd0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:44.705916 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-17b4d0f8-71b2-4a48-9fe5-ed070e2abbd0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:44.731804 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b4d0f8-71b2-4a48-9fe5-ed070e2abbd0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:44.731804 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b4d0f8-71b2-4a48-9fe5-ed070e2abbd0 service placement] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:44.744292 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1126/2271] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:43:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:43:47.341338 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=45 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:49.480188 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e80b955-9a70-4ac9-ad02-99f0912ec6d7 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:49.560281 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e80b955-9a70-4ac9-ad02-99f0912ec6d7 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:49.565499 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79a7193-f84b-4996-94fc-80d7872eaa3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:49.566153 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79a7193-f84b-4996-94fc-80d7872eaa3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:49.566438 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79a7193-f84b-4996-94fc-80d7872eaa3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:49.566719 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79a7193-f84b-4996-94fc-80d7872eaa3e admin admin] PATH_INFO: `/v3/users/325314efd6384e8dbd59473e73595d57` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:49.568020 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79a7193-f84b-4996-94fc-80d7872eaa3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=325314efd6384e8dbd59473e73595d57)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:49.571075 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79a7193-f84b-4996-94fc-80d7872eaa3e admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:49.812808 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-e79a7193-f84b-4996-94fc-80d7872eaa3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '325314efd6384e8dbd59473e73595d57'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:49.818794 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-e79a7193-f84b-4996-94fc-80d7872eaa3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '325314efd6384e8dbd59473e73595d57'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:49.818794 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-e79a7193-f84b-4996-94fc-80d7872eaa3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '325314efd6384e8dbd59473e73595d57'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:49.935734 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-e79a7193-f84b-4996-94fc-80d7872eaa3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '325314efd6384e8dbd59473e73595d57'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:49.963244 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-e79a7193-f84b-4996-94fc-80d7872eaa3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '325314efd6384e8dbd59473e73595d57'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:49.986296 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1146/2272] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:43:49 2022] DELETE /identity/v3/users/325314efd6384e8dbd59473e73595d57 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:43:49.997315 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e4f1a4-27bb-4216-bdd6-3fbe684ff16b None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:50.001881 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e4f1a4-27bb-4216-bdd6-3fbe684ff16b None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:50.001881 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e4f1a4-27bb-4216-bdd6-3fbe684ff16b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:50.018367 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-73e4f1a4-27bb-4216-bdd6-3fbe684ff16b None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:50.019952 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-73e4f1a4-27bb-4216-bdd6-3fbe684ff16b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:50.043170 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:50.102375 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1127/2273] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:43:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:43:50.112781 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79a7193-f84b-4996-94fc-80d7872eaa3e admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:50.114323 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-e79a7193-f84b-4996-94fc-80d7872eaa3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:50.185433 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79a7193-f84b-4996-94fc-80d7872eaa3e admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:50.189275 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d88093-7410-4dee-b0a5-d3a70e7f712f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:50.189918 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d88093-7410-4dee-b0a5-d3a70e7f712f service neutron] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:50.190201 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d88093-7410-4dee-b0a5-d3a70e7f712f service neutron] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:50.190474 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d88093-7410-4dee-b0a5-d3a70e7f712f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:50.202875 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d88093-7410-4dee-b0a5-d3a70e7f712f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:50.205019 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d88093-7410-4dee-b0a5-d3a70e7f712f service neutron] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:50.218553 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1147/2274] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:43:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:43:50.743329 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e4f1a4-27bb-4216-bdd6-3fbe684ff16b None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:50.744942 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-73e4f1a4-27bb-4216-bdd6-3fbe684ff16b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:50.758817 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e4f1a4-27bb-4216-bdd6-3fbe684ff16b None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:50.763710 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78496873-e05c-4eef-bc86-a29edfceb89a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:50.764411 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78496873-e05c-4eef-bc86-a29edfceb89a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:50.764711 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78496873-e05c-4eef-bc86-a29edfceb89a admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:50.765150 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78496873-e05c-4eef-bc86-a29edfceb89a admin admin] PATH_INFO: `/v3/projects/8519ae6912194be3ab89ddf8c83e9672` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:50.766289 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78496873-e05c-4eef-bc86-a29edfceb89a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8519ae6912194be3ab89ddf8c83e9672)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:50.768176 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78496873-e05c-4eef-bc86-a29edfceb89a admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:50.918117 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-78496873-e05c-4eef-bc86-a29edfceb89a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8519ae6912194be3ab89ddf8c83e9672', 'request_id': 'req-78496873-e05c-4eef-bc86-a29edfceb89a'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:50.940835 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-78496873-e05c-4eef-bc86-a29edfceb89a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8519ae6912194be3ab89ddf8c83e9672', 'request_id': 'req-78496873-e05c-4eef-bc86-a29edfceb89a'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:51.032304 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-78496873-e05c-4eef-bc86-a29edfceb89a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8519ae6912194be3ab89ddf8c83e9672', 'request_id': 'req-78496873-e05c-4eef-bc86-a29edfceb89a'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:51.046986 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-78496873-e05c-4eef-bc86-a29edfceb89a admin admin] The token cache is being invalidate because project 8519ae6912194be3ab89ddf8c83e9672 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:43:51.047694 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-78496873-e05c-4eef-bc86-a29edfceb89a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:51.048665 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-78496873-e05c-4eef-bc86-a29edfceb89a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:51.050839 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1128/2275] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:43:50 2022] DELETE /identity/v3/projects/8519ae6912194be3ab89ddf8c83e9672 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:43:55.530537 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b42125-2ed2-4ef4-886f-2bec27c8298b None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:55.531063 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b42125-2ed2-4ef4-886f-2bec27c8298b None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:55.531272 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b42125-2ed2-4ef4-886f-2bec27c8298b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:55.551688 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-d1b42125-2ed2-4ef4-886f-2bec27c8298b None None] MFA Rules not processed for user `ea70acd744994250a2905b5dd7163dfb`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:43:55.552894 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-d1b42125-2ed2-4ef4-886f-2bec27c8298b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:55.571477 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:55.612491 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1148/2276] 10.222.0.47 () {58 vars in 967 bytes} [Wed Aug 24 06:43:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:43:55.626113 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78496873-e05c-4eef-bc86-a29edfceb89a admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:55.627758 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-78496873-e05c-4eef-bc86-a29edfceb89a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:55.702870 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78496873-e05c-4eef-bc86-a29edfceb89a admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:55.707762 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6556c7d6-baef-43b8-9add-3999b56182b9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f1cbaf2cd6427585f298c9d58c319a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:55.708407 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6556c7d6-baef-43b8-9add-3999b56182b9 service swift] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:55.708689 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6556c7d6-baef-43b8-9add-3999b56182b9 service swift] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:55.708963 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6556c7d6-baef-43b8-9add-3999b56182b9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:55.718375 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6556c7d6-baef-43b8-9add-3999b56182b9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:55.720233 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6556c7d6-baef-43b8-9add-3999b56182b9 service swift] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:55.729428 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1129/2277] 10.222.0.47 () {60 vars in 1384 bytes} [Wed Aug 24 06:43:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 06:43:59.159836 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b42125-2ed2-4ef4-886f-2bec27c8298b None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:59.161792 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-d1b42125-2ed2-4ef4-886f-2bec27c8298b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:43:59.228868 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b42125-2ed2-4ef4-886f-2bec27c8298b None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:59.233367 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e97e885-68a3-4b95-916f-ea274b352c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:59.233994 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e97e885-68a3-4b95-916f-ea274b352c80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:59.234274 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e97e885-68a3-4b95-916f-ea274b352c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:59.234544 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e97e885-68a3-4b95-916f-ea274b352c80 admin admin] PATH_INFO: `/v3/users/bb4a45146170497098d2071618ea6469` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:59.235676 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e97e885-68a3-4b95-916f-ea274b352c80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb4a45146170497098d2071618ea6469)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:59.237726 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e97e885-68a3-4b95-916f-ea274b352c80 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:43:59.458382 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-4e97e885-68a3-4b95-916f-ea274b352c80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb4a45146170497098d2071618ea6469'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:59.459011 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-4e97e885-68a3-4b95-916f-ea274b352c80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb4a45146170497098d2071618ea6469'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:59.460260 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-4e97e885-68a3-4b95-916f-ea274b352c80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb4a45146170497098d2071618ea6469'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:59.541271 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-4e97e885-68a3-4b95-916f-ea274b352c80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb4a45146170497098d2071618ea6469'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:59.554825 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-4e97e885-68a3-4b95-916f-ea274b352c80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb4a45146170497098d2071618ea6469'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:43:59.567792 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1149/2278] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:43:59 2022] DELETE /identity/v3/users/bb4a45146170497098d2071618ea6469 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:43:59.818570 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6556c7d6-baef-43b8-9add-3999b56182b9 service swift] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:43:59.832775 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6556c7d6-baef-43b8-9add-3999b56182b9 service swift] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:43:59.870357 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:43:59.898982 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170fe17f-29e2-424b-8e31-452105dc9ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:43:59.899652 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170fe17f-29e2-424b-8e31-452105dc9ee9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:43:59.899974 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170fe17f-29e2-424b-8e31-452105dc9ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:43:59.900252 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170fe17f-29e2-424b-8e31-452105dc9ee9 admin admin] PATH_INFO: `/v3/projects/2affaa1f6bef4cdcb559126588333d96` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:43:59.901447 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170fe17f-29e2-424b-8e31-452105dc9ee9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2affaa1f6bef4cdcb559126588333d96)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:43:59.903392 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170fe17f-29e2-424b-8e31-452105dc9ee9 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:00.022148 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-170fe17f-29e2-424b-8e31-452105dc9ee9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2affaa1f6bef4cdcb559126588333d96', 'request_id': 'req-170fe17f-29e2-424b-8e31-452105dc9ee9'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:00.028329 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-170fe17f-29e2-424b-8e31-452105dc9ee9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2affaa1f6bef4cdcb559126588333d96', 'request_id': 'req-170fe17f-29e2-424b-8e31-452105dc9ee9'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:00.135702 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-170fe17f-29e2-424b-8e31-452105dc9ee9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2affaa1f6bef4cdcb559126588333d96', 'request_id': 'req-170fe17f-29e2-424b-8e31-452105dc9ee9'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:00.147850 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-170fe17f-29e2-424b-8e31-452105dc9ee9 admin admin] The token cache is being invalidate because project 2affaa1f6bef4cdcb559126588333d96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:44:00.148176 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-170fe17f-29e2-424b-8e31-452105dc9ee9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:00.148869 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-170fe17f-29e2-424b-8e31-452105dc9ee9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:00.150412 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1130/2279] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:43:59 2022] DELETE /identity/v3/projects/2affaa1f6bef4cdcb559126588333d96 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:44:00.156449 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec06c9a5-ebda-4567-8606-827ab1243047 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:00.156785 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec06c9a5-ebda-4567-8606-827ab1243047 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:00.157013 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec06c9a5-ebda-4567-8606-827ab1243047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:00.178074 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-ec06c9a5-ebda-4567-8606-827ab1243047 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:44:00.180008 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-ec06c9a5-ebda-4567-8606-827ab1243047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:00.224468 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:44:00.244871 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1150/2280] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:44:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:44:00.252670 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9087b043-21d0-4c52-bd61-53667ef054a8 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:00.252959 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9087b043-21d0-4c52-bd61-53667ef054a8 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:00.253262 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9087b043-21d0-4c52-bd61-53667ef054a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:00.273532 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-9087b043-21d0-4c52-bd61-53667ef054a8 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:44:00.274928 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-9087b043-21d0-4c52-bd61-53667ef054a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:00.288377 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1131/2281] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:44:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:44:00.298323 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec06c9a5-ebda-4567-8606-827ab1243047 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:00.309226 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec06c9a5-ebda-4567-8606-827ab1243047 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:00.318347 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88c99c1-7f73-4304-b2c1-e11a3341553c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:00.318347 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88c99c1-7f73-4304-b2c1-e11a3341553c admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:00.318347 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88c99c1-7f73-4304-b2c1-e11a3341553c admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:00.318347 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88c99c1-7f73-4304-b2c1-e11a3341553c admin admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:00.318705 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88c99c1-7f73-4304-b2c1-e11a3341553c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:44:00.318705 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88c99c1-7f73-4304-b2c1-e11a3341553c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:00.318705 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88c99c1-7f73-4304-b2c1-e11a3341553c admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:00.336257 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1151/2282] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 24 06:44:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:44:00.341335 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9087b043-21d0-4c52-bd61-53667ef054a8 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:00.347704 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9087b043-21d0-4c52-bd61-53667ef054a8 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:00.352654 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66966c0a-f45d-4695-8021-19e92a3919ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:00.353298 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66966c0a-f45d-4695-8021-19e92a3919ed admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:00.353585 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66966c0a-f45d-4695-8021-19e92a3919ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:00.353861 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66966c0a-f45d-4695-8021-19e92a3919ed admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:00.354499 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66966c0a-f45d-4695-8021-19e92a3919ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:00.356419 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66966c0a-f45d-4695-8021-19e92a3919ed admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:00.450019 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1132/2283] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:44:00 2022] POST /identity/v3/projects => generated 375 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:44:00.451303 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88c99c1-7f73-4304-b2c1-e11a3341553c admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:00.461750 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88c99c1-7f73-4304-b2c1-e11a3341553c admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:00.520742 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f1e1a8-e007-4fff-95c3-8c887e9d68cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:00.521329 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f1e1a8-e007-4fff-95c3-8c887e9d68cb admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:00.521617 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f1e1a8-e007-4fff-95c3-8c887e9d68cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:00.521894 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f1e1a8-e007-4fff-95c3-8c887e9d68cb admin admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:00.522771 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f1e1a8-e007-4fff-95c3-8c887e9d68cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:00.524730 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f1e1a8-e007-4fff-95c3-8c887e9d68cb admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:00.599535 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1152/2284] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:44:00 2022] POST /identity/v3/users => generated 288 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:44:00.603853 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66966c0a-f45d-4695-8021-19e92a3919ed admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:00.614204 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66966c0a-f45d-4695-8021-19e92a3919ed admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:00.620120 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c2ba0a-5d37-4dc3-9cd2-e53005c43ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:00.620764 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c2ba0a-5d37-4dc3-9cd2-e53005c43ace admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:00.621047 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c2ba0a-5d37-4dc3-9cd2-e53005c43ace admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:00.621361 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c2ba0a-5d37-4dc3-9cd2-e53005c43ace admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:00.622039 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c2ba0a-5d37-4dc3-9cd2-e53005c43ace admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:00.623666 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c2ba0a-5d37-4dc3-9cd2-e53005c43ace admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:00.642806 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1133/2285] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:44:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:44:00.645938 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f1e1a8-e007-4fff-95c3-8c887e9d68cb admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:00.649964 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f1e1a8-e007-4fff-95c3-8c887e9d68cb admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:00.654744 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71909cf-6701-4e7e-bffe-8b8e4649c02b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:00.655361 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71909cf-6701-4e7e-bffe-8b8e4649c02b admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:00.655643 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71909cf-6701-4e7e-bffe-8b8e4649c02b admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:00.655919 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71909cf-6701-4e7e-bffe-8b8e4649c02b admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:00.656569 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71909cf-6701-4e7e-bffe-8b8e4649c02b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:00.658196 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71909cf-6701-4e7e-bffe-8b8e4649c02b admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:00.673491 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1153/2286] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:44:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:44:00.674703 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c2ba0a-5d37-4dc3-9cd2-e53005c43ace admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:00.685759 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c2ba0a-5d37-4dc3-9cd2-e53005c43ace admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:00.690768 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c10f5e-742c-4750-9bab-e2dcfdca5ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:00.691434 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c10f5e-742c-4750-9bab-e2dcfdca5ce8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:00.691714 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c10f5e-742c-4750-9bab-e2dcfdca5ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:00.691995 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c10f5e-742c-4750-9bab-e2dcfdca5ce8 admin admin] PATH_INFO: `/v3/projects/292886d5ade34571bc39c5707171509d/users/9ba76090d89f4b5d842fb245cc95984b/roles/e1f9b167c51c4a388cbc9594db8a465d` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:00.713669 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c10f5e-742c-4750-9bab-e2dcfdca5ce8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=292886d5ade34571bc39c5707171509d, user_id=9ba76090d89f4b5d842fb245cc95984b, role_id=e1f9b167c51c4a388cbc9594db8a465d)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:00.714441 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c10f5e-742c-4750-9bab-e2dcfdca5ce8 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:00.840039 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1134/2287] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:44:00 2022] PUT /identity/v3/projects/292886d5ade34571bc39c5707171509d/users/9ba76090d89f4b5d842fb245cc95984b/roles/e1f9b167c51c4a388cbc9594db8a465d => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:44:00.851256 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445226b0-e78b-4a7d-8b62-4603139cff0b None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:00.851542 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445226b0-e78b-4a7d-8b62-4603139cff0b None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:00.851819 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445226b0-e78b-4a7d-8b62-4603139cff0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:00.881105 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-445226b0-e78b-4a7d-8b62-4603139cff0b None None] MFA Rules not processed for user `9ba76090d89f4b5d842fb245cc95984b`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:44:00.883105 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-445226b0-e78b-4a7d-8b62-4603139cff0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:00.952120 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1154/2288] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:44:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:44:00.964497 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c10f5e-742c-4750-9bab-e2dcfdca5ce8 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:00.966274 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-51c10f5e-742c-4750-9bab-e2dcfdca5ce8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:01.027436 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c10f5e-742c-4750-9bab-e2dcfdca5ce8 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:01.031095 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81fa3bf-0de2-44ff-be2a-6abe0cba10f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36639dce4f97412f89d71367f650b901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:01.031740 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81fa3bf-0de2-44ff-be2a-6abe0cba10f3 service cinder] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:01.032017 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81fa3bf-0de2-44ff-be2a-6abe0cba10f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:01.032285 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81fa3bf-0de2-44ff-be2a-6abe0cba10f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:01.041658 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81fa3bf-0de2-44ff-be2a-6abe0cba10f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:01.043146 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81fa3bf-0de2-44ff-be2a-6abe0cba10f3 service cinder] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:01.051893 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1135/2289] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:44:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:44:03.860613 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5fc3d1-686e-4a6e-9141-a96d42d4fe43 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:03.861154 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5fc3d1-686e-4a6e-9141-a96d42d4fe43 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:03.861453 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5fc3d1-686e-4a6e-9141-a96d42d4fe43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:03.889199 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-4d5fc3d1-686e-4a6e-9141-a96d42d4fe43 None None] MFA Rules not processed for user `9ba76090d89f4b5d842fb245cc95984b`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:44:03.889199 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-4d5fc3d1-686e-4a6e-9141-a96d42d4fe43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:03.892003 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1155/2290] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:44:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:44:09.672970 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81fa3bf-0de2-44ff-be2a-6abe0cba10f3 service cinder] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:09.674600 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-b81fa3bf-0de2-44ff-be2a-6abe0cba10f3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:09.682985 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:44:09.752647 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81fa3bf-0de2-44ff-be2a-6abe0cba10f3 service cinder] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:09.756363 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980caedd-e974-4ac7-8c2f-c9e144a0d400 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:09.756976 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980caedd-e974-4ac7-8c2f-c9e144a0d400 service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:09.757575 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980caedd-e974-4ac7-8c2f-c9e144a0d400 service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:09.757575 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980caedd-e974-4ac7-8c2f-c9e144a0d400 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:09.760121 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-980caedd-e974-4ac7-8c2f-c9e144a0d400 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:09.844314 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980caedd-e974-4ac7-8c2f-c9e144a0d400 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:09.846268 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980caedd-e974-4ac7-8c2f-c9e144a0d400 service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:09.858258 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1136/2291] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:44:09 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:44:13.876712 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=81,keystone:INSERT=3 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:44:14.576762 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5fc3d1-686e-4a6e-9141-a96d42d4fe43 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:14.588528 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5fc3d1-686e-4a6e-9141-a96d42d4fe43 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:14.594131 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b175d1-450f-4298-85d6-0a8d805c914a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:14.595231 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b175d1-450f-4298-85d6-0a8d805c914a service neutron] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:14.595945 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b175d1-450f-4298-85d6-0a8d805c914a service neutron] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:14.596374 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b175d1-450f-4298-85d6-0a8d805c914a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:14.599051 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-94b175d1-450f-4298-85d6-0a8d805c914a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:14.699775 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b175d1-450f-4298-85d6-0a8d805c914a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:14.701473 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b175d1-450f-4298-85d6-0a8d805c914a service neutron] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:14.712930 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1156/2292] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:44:14 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:44:20.223533 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=48 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:44:24.707283 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=30 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:44:24.938574 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a58af71-4bda-4bdb-a71d-a0b9091d8bfd None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:24.939130 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a58af71-4bda-4bdb-a71d-a0b9091d8bfd None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:24.939257 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a58af71-4bda-4bdb-a71d-a0b9091d8bfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:24.960115 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-8a58af71-4bda-4bdb-a71d-a0b9091d8bfd None None] MFA Rules not processed for user `02ad6cdc4ffe4917a7bbf6814297f980`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:44:24.961482 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-8a58af71-4bda-4bdb-a71d-a0b9091d8bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:25.020654 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1137/2293] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:44:24 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:44:33.357369 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f43e03-f590-40ea-92f5-03b649d8a03a None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:33.357844 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f43e03-f590-40ea-92f5-03b649d8a03a None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:33.357943 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f43e03-f590-40ea-92f5-03b649d8a03a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:33.384202 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-46f43e03-f590-40ea-92f5-03b649d8a03a None None] MFA Rules not processed for user `026744699ed54dce8dc240e44e8aefb3`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:44:33.384202 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-46f43e03-f590-40ea-92f5-03b649d8a03a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:33.463720 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1157/2294] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:44:33 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:44:35.013824 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=25 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:44:35.154628 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e6936e-a8e2-475f-92d5-3d5c11985fc1 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:35.154911 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e6936e-a8e2-475f-92d5-3d5c11985fc1 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:35.155200 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e6936e-a8e2-475f-92d5-3d5c11985fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:35.176495 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-40e6936e-a8e2-475f-92d5-3d5c11985fc1 None None] MFA Rules not processed for user `ea70acd744994250a2905b5dd7163dfb`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:44:35.177764 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-40e6936e-a8e2-475f-92d5-3d5c11985fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:35.262435 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1138/2295] 10.222.0.47 () {58 vars in 1009 bytes} [Wed Aug 24 06:44:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:44:35.263348 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f43e03-f590-40ea-92f5-03b649d8a03a None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:35.263348 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-46f43e03-f590-40ea-92f5-03b649d8a03a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:35.337745 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f43e03-f590-40ea-92f5-03b649d8a03a None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:35.343541 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a076df84-4ad7-4a1c-abbb-d6838fb22443 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f1cbaf2cd6427585f298c9d58c319a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:35.344159 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a076df84-4ad7-4a1c-abbb-d6838fb22443 service swift] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:35.344440 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a076df84-4ad7-4a1c-abbb-d6838fb22443 service swift] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:35.344720 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a076df84-4ad7-4a1c-abbb-d6838fb22443 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:35.354667 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a076df84-4ad7-4a1c-abbb-d6838fb22443 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:35.358800 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a076df84-4ad7-4a1c-abbb-d6838fb22443 service swift] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:35.368420 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1158/2296] 10.222.0.47 () {60 vars in 1384 bytes} [Wed Aug 24 06:44:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 06:44:36.417745 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d86ea7c-39a9-4e6f-9a9f-b8980de36f1c None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:36.418210 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d86ea7c-39a9-4e6f-9a9f-b8980de36f1c None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:36.418308 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d86ea7c-39a9-4e6f-9a9f-b8980de36f1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:36.441332 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-1d86ea7c-39a9-4e6f-9a9f-b8980de36f1c None None] MFA Rules not processed for user `ea70acd744994250a2905b5dd7163dfb`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:44:36.442541 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-1d86ea7c-39a9-4e6f-9a9f-b8980de36f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:36.450011 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1139/2297] 10.222.0.47 () {58 vars in 967 bytes} [Wed Aug 24 06:44:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:44:36.463340 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a076df84-4ad7-4a1c-abbb-d6838fb22443 service swift] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:36.474667 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a076df84-4ad7-4a1c-abbb-d6838fb22443 service swift] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:36.479942 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29716cb-060b-4bc4-97f8-8cc972bf397c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f1cbaf2cd6427585f298c9d58c319a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:36.480538 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29716cb-060b-4bc4-97f8-8cc972bf397c service swift] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:36.480843 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29716cb-060b-4bc4-97f8-8cc972bf397c service swift] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:36.481123 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29716cb-060b-4bc4-97f8-8cc972bf397c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:36.490788 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29716cb-060b-4bc4-97f8-8cc972bf397c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:36.492344 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29716cb-060b-4bc4-97f8-8cc972bf397c service swift] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:36.502688 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1159/2298] 10.222.0.47 () {60 vars in 1384 bytes} [Wed Aug 24 06:44:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 06:44:37.228210 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec33604-cab0-4253-b495-4c9dae9eae13 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:37.228736 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec33604-cab0-4253-b495-4c9dae9eae13 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:37.229289 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec33604-cab0-4253-b495-4c9dae9eae13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:37.258456 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-fec33604-cab0-4253-b495-4c9dae9eae13 None None] MFA Rules not processed for user `ea70acd744994250a2905b5dd7163dfb`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:44:37.259933 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-fec33604-cab0-4253-b495-4c9dae9eae13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:37.268843 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1140/2299] 10.222.0.47 () {58 vars in 1009 bytes} [Wed Aug 24 06:44:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:44:37.283838 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29716cb-060b-4bc4-97f8-8cc972bf397c service swift] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:37.296215 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29716cb-060b-4bc4-97f8-8cc972bf397c service swift] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:37.304097 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4cc3f8-d312-4b61-915c-8f9fb1b067f4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f1cbaf2cd6427585f298c9d58c319a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:37.304713 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4cc3f8-d312-4b61-915c-8f9fb1b067f4 service swift] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:37.305006 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4cc3f8-d312-4b61-915c-8f9fb1b067f4 service swift] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:37.305294 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4cc3f8-d312-4b61-915c-8f9fb1b067f4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:37.314571 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4cc3f8-d312-4b61-915c-8f9fb1b067f4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:37.316189 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4cc3f8-d312-4b61-915c-8f9fb1b067f4 service swift] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:37.333392 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1160/2300] 10.222.0.47 () {60 vars in 1384 bytes} [Wed Aug 24 06:44:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 06:44:38.076522 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305582a3-49f8-414e-99b1-945a9b46ed5a None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:38.077338 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305582a3-49f8-414e-99b1-945a9b46ed5a None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:38.077338 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305582a3-49f8-414e-99b1-945a9b46ed5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:38.098905 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-305582a3-49f8-414e-99b1-945a9b46ed5a None None] MFA Rules not processed for user `ea70acd744994250a2905b5dd7163dfb`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:44:38.100433 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-305582a3-49f8-414e-99b1-945a9b46ed5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:38.108604 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1141/2301] 10.222.0.47 () {58 vars in 967 bytes} [Wed Aug 24 06:44:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:44:38.124195 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4cc3f8-d312-4b61-915c-8f9fb1b067f4 service swift] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:38.136515 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4cc3f8-d312-4b61-915c-8f9fb1b067f4 service swift] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:38.143625 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257699eb-e51b-41bb-b9a5-e6a69c91e94d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f1cbaf2cd6427585f298c9d58c319a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:38.144506 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257699eb-e51b-41bb-b9a5-e6a69c91e94d service swift] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:38.144930 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257699eb-e51b-41bb-b9a5-e6a69c91e94d service swift] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:38.145439 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257699eb-e51b-41bb-b9a5-e6a69c91e94d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:38.157122 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257699eb-e51b-41bb-b9a5-e6a69c91e94d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:38.159593 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257699eb-e51b-41bb-b9a5-e6a69c91e94d service swift] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:38.171584 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1161/2302] 10.222.0.47 () {60 vars in 1384 bytes} [Wed Aug 24 06:44:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 06:44:41.462180 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305582a3-49f8-414e-99b1-945a9b46ed5a None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:41.537826 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305582a3-49f8-414e-99b1-945a9b46ed5a None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:41.542967 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b476940-46c6-49b4-ab5f-3b8e3bed966a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:41.543617 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b476940-46c6-49b4-ab5f-3b8e3bed966a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:41.544061 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b476940-46c6-49b4-ab5f-3b8e3bed966a admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:41.544425 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b476940-46c6-49b4-ab5f-3b8e3bed966a admin admin] PATH_INFO: `/v3/users/9ba76090d89f4b5d842fb245cc95984b` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:41.545949 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b476940-46c6-49b4-ab5f-3b8e3bed966a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ba76090d89f4b5d842fb245cc95984b)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:41.547882 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b476940-46c6-49b4-ab5f-3b8e3bed966a admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:41.729481 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-4b476940-46c6-49b4-ab5f-3b8e3bed966a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ba76090d89f4b5d842fb245cc95984b'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:41.730250 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-4b476940-46c6-49b4-ab5f-3b8e3bed966a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ba76090d89f4b5d842fb245cc95984b'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:41.731214 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-4b476940-46c6-49b4-ab5f-3b8e3bed966a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ba76090d89f4b5d842fb245cc95984b'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:41.741298 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-4b476940-46c6-49b4-ab5f-3b8e3bed966a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ba76090d89f4b5d842fb245cc95984b'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:41.788193 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-4b476940-46c6-49b4-ab5f-3b8e3bed966a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ba76090d89f4b5d842fb245cc95984b'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:41.807405 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1142/2303] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:44:41 2022] DELETE /identity/v3/users/9ba76090d89f4b5d842fb245cc95984b => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:44:41.812526 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfe9ee6-aa66-4e4d-a727-820e8d4482d5 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:41.813021 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfe9ee6-aa66-4e4d-a727-820e8d4482d5 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:41.813115 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfe9ee6-aa66-4e4d-a727-820e8d4482d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:41.834698 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-3dfe9ee6-aa66-4e4d-a727-820e8d4482d5 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:44:41.835621 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-3dfe9ee6-aa66-4e4d-a727-820e8d4482d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:41.890547 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:44:41.909460 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1162/2304] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:44:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:44:41.922204 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b476940-46c6-49b4-ab5f-3b8e3bed966a admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:41.924143 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-4b476940-46c6-49b4-ab5f-3b8e3bed966a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:41.931630 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:44:41.993885 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b476940-46c6-49b4-ab5f-3b8e3bed966a admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:41.999964 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ac185f-34aa-4b79-8ba0-128fdc158d6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:42.000770 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac185f-34aa-4b79-8ba0-128fdc158d6f service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:42.001055 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac185f-34aa-4b79-8ba0-128fdc158d6f service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:42.001364 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ac185f-34aa-4b79-8ba0-128fdc158d6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:42.011509 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ac185f-34aa-4b79-8ba0-128fdc158d6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:42.013137 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ac185f-34aa-4b79-8ba0-128fdc158d6f service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:42.025042 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1143/2305] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:44:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:44:42.452582 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfe9ee6-aa66-4e4d-a727-820e8d4482d5 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:42.454063 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-3dfe9ee6-aa66-4e4d-a727-820e8d4482d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:42.467883 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfe9ee6-aa66-4e4d-a727-820e8d4482d5 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:42.472953 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4420a6-88da-4785-9e55-912a307a9700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:42.473797 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4420a6-88da-4785-9e55-912a307a9700 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:42.474089 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4420a6-88da-4785-9e55-912a307a9700 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:42.474366 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4420a6-88da-4785-9e55-912a307a9700 admin admin] PATH_INFO: `/v3/projects/292886d5ade34571bc39c5707171509d` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:42.475514 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4420a6-88da-4785-9e55-912a307a9700 admin admin] RBAC: Authorizing `identity:delete_project(project_id=292886d5ade34571bc39c5707171509d)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:42.477484 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4420a6-88da-4785-9e55-912a307a9700 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:42.588957 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-8a4420a6-88da-4785-9e55-912a307a9700 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '292886d5ade34571bc39c5707171509d', 'request_id': 'req-8a4420a6-88da-4785-9e55-912a307a9700'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:42.656094 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-8a4420a6-88da-4785-9e55-912a307a9700 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '292886d5ade34571bc39c5707171509d', 'request_id': 'req-8a4420a6-88da-4785-9e55-912a307a9700'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:42.663339 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-8a4420a6-88da-4785-9e55-912a307a9700 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '292886d5ade34571bc39c5707171509d', 'request_id': 'req-8a4420a6-88da-4785-9e55-912a307a9700'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:42.675301 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-8a4420a6-88da-4785-9e55-912a307a9700 admin admin] The token cache is being invalidate because project 292886d5ade34571bc39c5707171509d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81723) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:44:42.675591 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-8a4420a6-88da-4785-9e55-912a307a9700 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:42.676315 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-8a4420a6-88da-4785-9e55-912a307a9700 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:44:42.678179 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1163/2306] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:44:42 2022] DELETE /identity/v3/projects/292886d5ade34571bc39c5707171509d => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:44:52.069350 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=24 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:44:52.672210 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=29,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:44:56.599628 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc824653-d7d4-439d-a728-784ab088c780 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:56.600122 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc824653-d7d4-439d-a728-784ab088c780 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:56.600300 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc824653-d7d4-439d-a728-784ab088c780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:56.621332 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-bc824653-d7d4-439d-a728-784ab088c780 None None] MFA Rules not processed for user `ea70acd744994250a2905b5dd7163dfb`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:44:56.622622 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-bc824653-d7d4-439d-a728-784ab088c780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:56.682841 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1144/2307] 10.222.0.47 () {58 vars in 1009 bytes} [Wed Aug 24 06:44:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:44:56.695367 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4420a6-88da-4785-9e55-912a307a9700 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:44:56.696882 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-8a4420a6-88da-4785-9e55-912a307a9700 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:44:56.769750 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4420a6-88da-4785-9e55-912a307a9700 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:44:56.775210 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f182112-fc13-44e1-b5ae-47da00bde1a5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f1cbaf2cd6427585f298c9d58c319a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:44:56.775823 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f182112-fc13-44e1-b5ae-47da00bde1a5 service swift] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:44:56.776102 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f182112-fc13-44e1-b5ae-47da00bde1a5 service swift] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:44:56.776429 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f182112-fc13-44e1-b5ae-47da00bde1a5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:44:56.785773 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f182112-fc13-44e1-b5ae-47da00bde1a5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:44:56.787357 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f182112-fc13-44e1-b5ae-47da00bde1a5 service swift] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:44:56.797171 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1164/2308] 10.222.0.47 () {60 vars in 1384 bytes} [Wed Aug 24 06:44:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 06:45:06.675701 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=25 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:45:06.792532 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=32 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:45:15.511258 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52087879-1543-4bc2-b816-48545f0ee954 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:15.511730 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52087879-1543-4bc2-b816-48545f0ee954 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:15.511811 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52087879-1543-4bc2-b816-48545f0ee954 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:15.513289 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1145/2309] 10.222.0.47 () {54 vars in 921 bytes} [Wed Aug 24 06:45:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:45:15.519746 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a8f717-2a19-41f4-aeac-022702d1acf1 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:15.520241 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a8f717-2a19-41f4-aeac-022702d1acf1 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:15.520336 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a8f717-2a19-41f4-aeac-022702d1acf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:15.541132 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-25a8f717-2a19-41f4-aeac-022702d1acf1 None None] MFA Rules not processed for user `1f4f49c1b08c460f8ba6c451c9c92c2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:45:15.542745 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-25a8f717-2a19-41f4-aeac-022702d1acf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:45:15.621541 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1165/2310] 10.222.0.47 () {58 vars in 1018 bytes} [Wed Aug 24 06:45:15 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:45:15.633668 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52087879-1543-4bc2-b816-48545f0ee954 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:45:15.635416 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-52087879-1543-4bc2-b816-48545f0ee954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:45:15.695960 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52087879-1543-4bc2-b816-48545f0ee954 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:45:15.699472 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ee9951-7cf4-45ce-8d8e-a003d396c12e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b331b6ddfe46aeae7b52fcbbc27771', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:45:15.700092 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ee9951-7cf4-45ce-8d8e-a003d396c12e service glance] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:15.700377 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ee9951-7cf4-45ce-8d8e-a003d396c12e service glance] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:15.700653 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ee9951-7cf4-45ce-8d8e-a003d396c12e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:15.711551 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ee9951-7cf4-45ce-8d8e-a003d396c12e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:45:15.713002 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ee9951-7cf4-45ce-8d8e-a003d396c12e service glance] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:45:15.724911 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1146/2311] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:45:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:45:15.956584 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52a98cd-e702-4082-97b0-aca832e2f402 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:15.956780 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52a98cd-e702-4082-97b0-aca832e2f402 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:15.957044 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52a98cd-e702-4082-97b0-aca832e2f402 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:15.958440 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1166/2312] 10.222.0.47 () {54 vars in 921 bytes} [Wed Aug 24 06:45:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:45:15.963085 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae504f-0894-46d9-8dc6-3824ea06c325 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:15.963402 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae504f-0894-46d9-8dc6-3824ea06c325 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:15.963682 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae504f-0894-46d9-8dc6-3824ea06c325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:15.984497 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-58ae504f-0894-46d9-8dc6-3824ea06c325 None None] MFA Rules not processed for user `f0ba3f03ec15475ab6ebf64994ad742c`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:45:15.985803 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-58ae504f-0894-46d9-8dc6-3824ea06c325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:45:16.039821 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1147/2313] 10.222.0.47 () {58 vars in 1018 bytes} [Wed Aug 24 06:45:15 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:45:16.052579 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52a98cd-e702-4082-97b0-aca832e2f402 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:45:16.054439 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-e52a98cd-e702-4082-97b0-aca832e2f402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:45:16.068436 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52a98cd-e702-4082-97b0-aca832e2f402 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:45:16.072001 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f283b056-a1c1-46c7-8400-054f435ae7d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:45:16.075677 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f283b056-a1c1-46c7-8400-054f435ae7d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:16.075677 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f283b056-a1c1-46c7-8400-054f435ae7d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:16.075677 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f283b056-a1c1-46c7-8400-054f435ae7d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:16.081166 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f283b056-a1c1-46c7-8400-054f435ae7d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:45:16.082695 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f283b056-a1c1-46c7-8400-054f435ae7d7 service neutron] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:45:16.090701 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1167/2314] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:45:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:45:16.176689 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f039fe-c3a7-413a-b8ee-4bfec41b21e5 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:16.185559 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f039fe-c3a7-413a-b8ee-4bfec41b21e5 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:16.185559 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f039fe-c3a7-413a-b8ee-4bfec41b21e5 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:16.185559 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1148/2315] 10.222.0.47 () {54 vars in 921 bytes} [Wed Aug 24 06:45:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 06:45:16.185957 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f07ecd-182d-4342-9e5b-ebd37cccd047 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:16.185957 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f07ecd-182d-4342-9e5b-ebd37cccd047 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:16.185957 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f07ecd-182d-4342-9e5b-ebd37cccd047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:16.204745 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-b4f07ecd-182d-4342-9e5b-ebd37cccd047 None None] MFA Rules not processed for user `1f4f49c1b08c460f8ba6c451c9c92c2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:45:16.206224 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-b4f07ecd-182d-4342-9e5b-ebd37cccd047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:45:16.219920 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1168/2316] 10.222.0.47 () {58 vars in 1018 bytes} [Wed Aug 24 06:45:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:45:16.226991 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f039fe-c3a7-413a-b8ee-4bfec41b21e5 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:45:16.239077 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f039fe-c3a7-413a-b8ee-4bfec41b21e5 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:45:16.242941 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307523c3-2ff9-4d4f-8403-41c3dd7cb72c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:45:16.243559 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307523c3-2ff9-4d4f-8403-41c3dd7cb72c service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:16.243835 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307523c3-2ff9-4d4f-8403-41c3dd7cb72c service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:16.244121 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307523c3-2ff9-4d4f-8403-41c3dd7cb72c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:16.254799 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307523c3-2ff9-4d4f-8403-41c3dd7cb72c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:45:16.256252 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307523c3-2ff9-4d4f-8403-41c3dd7cb72c service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:45:16.266945 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1149/2317] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:45:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:45:20.058071 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e178703-1fcc-4a1a-8f76-00cda682369d None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:20.058585 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e178703-1fcc-4a1a-8f76-00cda682369d None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:20.058690 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e178703-1fcc-4a1a-8f76-00cda682369d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:20.079718 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-6e178703-1fcc-4a1a-8f76-00cda682369d None None] MFA Rules not processed for user `ea70acd744994250a2905b5dd7163dfb`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:45:20.081010 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-6e178703-1fcc-4a1a-8f76-00cda682369d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:45:20.088111 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1169/2318] 10.222.0.47 () {58 vars in 1009 bytes} [Wed Aug 24 06:45:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:45:20.100418 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307523c3-2ff9-4d4f-8403-41c3dd7cb72c service neutron] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:45:20.111487 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307523c3-2ff9-4d4f-8403-41c3dd7cb72c service neutron] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:45:20.117063 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5587767-ace3-438e-8c78-e55513894f97 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f1cbaf2cd6427585f298c9d58c319a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:45:20.117685 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5587767-ace3-438e-8c78-e55513894f97 service swift] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:20.117961 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5587767-ace3-438e-8c78-e55513894f97 service swift] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:20.118235 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5587767-ace3-438e-8c78-e55513894f97 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:20.126269 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5587767-ace3-438e-8c78-e55513894f97 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:45:20.128495 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5587767-ace3-438e-8c78-e55513894f97 service swift] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:45:20.136517 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1150/2319] 10.222.0.47 () {60 vars in 1384 bytes} [Wed Aug 24 06:45:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 06:45:30.073554 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=43 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:45:30.133388 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=61 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:45:31.028052 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e178703-1fcc-4a1a-8f76-00cda682369d None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:45:31.038116 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e178703-1fcc-4a1a-8f76-00cda682369d None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:45:31.041767 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175721cb-f912-4b91-a1e7-6a911e5ad319 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:45:31.042365 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175721cb-f912-4b91-a1e7-6a911e5ad319 service neutron] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:31.042647 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175721cb-f912-4b91-a1e7-6a911e5ad319 service neutron] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:31.042919 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175721cb-f912-4b91-a1e7-6a911e5ad319 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:31.044663 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-175721cb-f912-4b91-a1e7-6a911e5ad319 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:45:31.059140 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175721cb-f912-4b91-a1e7-6a911e5ad319 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:45:31.060569 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175721cb-f912-4b91-a1e7-6a911e5ad319 service neutron] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:45:31.072650 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1170/2320] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:45:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:45:32.286753 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5587767-ace3-438e-8c78-e55513894f97 service swift] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:45:32.289028 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-f5587767-ace3-438e-8c78-e55513894f97 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:45:32.312419 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5587767-ace3-438e-8c78-e55513894f97 service swift] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:45:32.312685 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a122c5a-e5db-4b92-b491-1a453fe80379 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:45:32.313255 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a122c5a-e5db-4b92-b491-1a453fe80379 service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:32.313510 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a122c5a-e5db-4b92-b491-1a453fe80379 service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:32.313841 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a122c5a-e5db-4b92-b491-1a453fe80379 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:32.315784 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-6a122c5a-e5db-4b92-b491-1a453fe80379 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:45:32.331924 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a122c5a-e5db-4b92-b491-1a453fe80379 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:45:32.333454 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a122c5a-e5db-4b92-b491-1a453fe80379 service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:45:32.347609 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1151/2321] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:45:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:45:41.066626 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=6 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:45:42.340429 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=6 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:45:51.312404 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175721cb-f912-4b91-a1e7-6a911e5ad319 service neutron] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:45:51.313809 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-175721cb-f912-4b91-a1e7-6a911e5ad319 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:45:51.328549 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175721cb-f912-4b91-a1e7-6a911e5ad319 service neutron] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:45:51.334577 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034aaf2c-4fce-4255-8cbc-078016601d31 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4f49c1b08c460f8ba6c451c9c92c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:45:51.335176 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034aaf2c-4fce-4255-8cbc-078016601d31 service nova] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:51.335454 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034aaf2c-4fce-4255-8cbc-078016601d31 service nova] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:51.335836 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034aaf2c-4fce-4255-8cbc-078016601d31 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:51.337804 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-034aaf2c-4fce-4255-8cbc-078016601d31 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:45:51.421049 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034aaf2c-4fce-4255-8cbc-078016601d31 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:45:51.424015 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034aaf2c-4fce-4255-8cbc-078016601d31 service nova] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:45:51.437055 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1171/2322] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:45:51 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:45:54.917882 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bdd005-d4ff-425c-b5a8-811292df2f28 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:54.918407 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bdd005-d4ff-425c-b5a8-811292df2f28 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:54.918528 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bdd005-d4ff-425c-b5a8-811292df2f28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:54.939544 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-03bdd005-d4ff-425c-b5a8-811292df2f28 None None] MFA Rules not processed for user `ea70acd744994250a2905b5dd7163dfb`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:45:54.940749 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-03bdd005-d4ff-425c-b5a8-811292df2f28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:45:54.949702 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1152/2323] 10.222.0.47 () {58 vars in 967 bytes} [Wed Aug 24 06:45:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:45:54.971928 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034aaf2c-4fce-4255-8cbc-078016601d31 service nova] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:45:54.983918 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034aaf2c-4fce-4255-8cbc-078016601d31 service nova] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:45:54.990714 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8dc61c-d06e-43b4-b3c1-c2b93f7b0cea service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f1cbaf2cd6427585f298c9d58c319a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:45:54.991361 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8dc61c-d06e-43b4-b3c1-c2b93f7b0cea service swift] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:45:54.991730 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8dc61c-d06e-43b4-b3c1-c2b93f7b0cea service swift] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:45:54.992332 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8dc61c-d06e-43b4-b3c1-c2b93f7b0cea service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:45:55.002056 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8dc61c-d06e-43b4-b3c1-c2b93f7b0cea service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:45:55.003831 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8dc61c-d06e-43b4-b3c1-c2b93f7b0cea service swift] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:45:55.013314 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1172/2324] 10.222.0.47 () {60 vars in 1384 bytes} [Wed Aug 24 06:45:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 06:46:04.934474 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=3 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:46:05.008973 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=34 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:46:27.696974 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bdd005-d4ff-425c-b5a8-811292df2f28 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:27.710284 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bdd005-d4ff-425c-b5a8-811292df2f28 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:27.716100 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281aed1e-3359-48c6-bf47-68f0590f5f18 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4f49c1b08c460f8ba6c451c9c92c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:27.716710 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281aed1e-3359-48c6-bf47-68f0590f5f18 service nova] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:27.716992 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281aed1e-3359-48c6-bf47-68f0590f5f18 service nova] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:27.717313 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281aed1e-3359-48c6-bf47-68f0590f5f18 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:27.719012 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-281aed1e-3359-48c6-bf47-68f0590f5f18 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:46:27.780765 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281aed1e-3359-48c6-bf47-68f0590f5f18 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:27.782378 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281aed1e-3359-48c6-bf47-68f0590f5f18 service nova] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:27.793035 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1153/2325] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:46:27 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:46:37.787039 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8dc61c-d06e-43b4-b3c1-c2b93f7b0cea service swift] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:37.789343 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=28 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:46:37.797693 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8dc61c-d06e-43b4-b3c1-c2b93f7b0cea service swift] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:37.801421 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d45c2e-e42c-4624-9ced-414faa55c116 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:37.802041 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d45c2e-e42c-4624-9ced-414faa55c116 service neutron] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:37.802325 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d45c2e-e42c-4624-9ced-414faa55c116 service neutron] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:37.802603 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d45c2e-e42c-4624-9ced-414faa55c116 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:37.804747 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-b7d45c2e-e42c-4624-9ced-414faa55c116 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:46:37.815156 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d45c2e-e42c-4624-9ced-414faa55c116 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:37.816607 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d45c2e-e42c-4624-9ced-414faa55c116 service neutron] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:37.825730 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1173/2326] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:46:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:46:45.335405 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281aed1e-3359-48c6-bf47-68f0590f5f18 service nova] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:45.347812 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281aed1e-3359-48c6-bf47-68f0590f5f18 service nova] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:45.355039 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb11f1b-e05e-4c08-ad1f-19427c388a06 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4f49c1b08c460f8ba6c451c9c92c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:45.355676 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb11f1b-e05e-4c08-ad1f-19427c388a06 service nova] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:45.356010 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb11f1b-e05e-4c08-ad1f-19427c388a06 service nova] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:45.356292 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb11f1b-e05e-4c08-ad1f-19427c388a06 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:45.358439 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-adb11f1b-e05e-4c08-ad1f-19427c388a06 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:46:45.425203 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb11f1b-e05e-4c08-ad1f-19427c388a06 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:45.426815 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb11f1b-e05e-4c08-ad1f-19427c388a06 service nova] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:45.437228 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1154/2327] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:46:45 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:46:47.822220 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=6 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:46:49.000805 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d45c2e-e42c-4624-9ced-414faa55c116 service neutron] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:46:49.015476 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d45c2e-e42c-4624-9ced-414faa55c116 service neutron] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:46:49.021528 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec9352f-7ca7-42b5-94b8-1ef8b1a21d3e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4f49c1b08c460f8ba6c451c9c92c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:46:49.022150 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec9352f-7ca7-42b5-94b8-1ef8b1a21d3e service nova] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:46:49.022433 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec9352f-7ca7-42b5-94b8-1ef8b1a21d3e service nova] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:46:49.022707 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec9352f-7ca7-42b5-94b8-1ef8b1a21d3e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:46:49.025404 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-bec9352f-7ca7-42b5-94b8-1ef8b1a21d3e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:46:49.042184 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec9352f-7ca7-42b5-94b8-1ef8b1a21d3e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:46:49.043785 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec9352f-7ca7-42b5-94b8-1ef8b1a21d3e service nova] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:46:49.066442 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1174/2328] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:46:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:46:55.433288 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=28 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:46:59.050077 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=6 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:47:03.497122 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dbfc38-caa7-4e19-8ab2-61421808c094 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:47:03.497689 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dbfc38-caa7-4e19-8ab2-61421808c094 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:47:03.497689 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dbfc38-caa7-4e19-8ab2-61421808c094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:47:03.517101 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-a3dbfc38-caa7-4e19-8ab2-61421808c094 None None] MFA Rules not processed for user `02ad6cdc4ffe4917a7bbf6814297f980`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:47:03.518343 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-a3dbfc38-caa7-4e19-8ab2-61421808c094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:47:03.525509 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1155/2329] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:47:03 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:47:03.839996 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec9352f-7ca7-42b5-94b8-1ef8b1a21d3e service nova] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:47:03.850058 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec9352f-7ca7-42b5-94b8-1ef8b1a21d3e service nova] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:47:03.854436 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9788beed-199e-414d-91fc-790f298c27e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:47:03.855095 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9788beed-199e-414d-91fc-790f298c27e1 service neutron] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:47:03.855380 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9788beed-199e-414d-91fc-790f298c27e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:47:03.855656 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9788beed-199e-414d-91fc-790f298c27e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:47:03.857436 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-9788beed-199e-414d-91fc-790f298c27e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:47:03.866968 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9788beed-199e-414d-91fc-790f298c27e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:47:03.868400 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9788beed-199e-414d-91fc-790f298c27e1 service neutron] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:47:03.877323 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1175/2330] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:47:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:47:05.317995 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dbfc38-caa7-4e19-8ab2-61421808c094 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:47:05.326306 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-a3dbfc38-caa7-4e19-8ab2-61421808c094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:47:05.427416 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dbfc38-caa7-4e19-8ab2-61421808c094 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:47:05.436244 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea5c710-5d6e-4420-89ad-be77745157c1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '306188d42dbd4b5b84082ab3842eaf9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:47:05.437246 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea5c710-5d6e-4420-89ad-be77745157c1 service placement] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:47:05.437814 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea5c710-5d6e-4420-89ad-be77745157c1 service placement] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:47:05.438279 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea5c710-5d6e-4420-89ad-be77745157c1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:47:05.440472 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-8ea5c710-5d6e-4420-89ad-be77745157c1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:47:05.454618 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea5c710-5d6e-4420-89ad-be77745157c1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:47:05.456372 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea5c710-5d6e-4420-89ad-be77745157c1 service placement] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:47:05.470027 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1156/2331] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:47:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:47:07.814387 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0057742e-8a6e-4c1a-9c15-79a639a63ec4 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:47:07.814970 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0057742e-8a6e-4c1a-9c15-79a639a63ec4 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:47:07.815088 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0057742e-8a6e-4c1a-9c15-79a639a63ec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:47:07.836343 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-0057742e-8a6e-4c1a-9c15-79a639a63ec4 None None] MFA Rules not processed for user `594e9324c44540568a81fd5fdb980efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:47:07.836928 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-0057742e-8a6e-4c1a-9c15-79a639a63ec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:47:07.896505 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1176/2332] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:47:07 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:47:14.744820 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4469ab94-0efc-4278-ab24-cc7892d1e678 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:47:14.745490 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4469ab94-0efc-4278-ab24-cc7892d1e678 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:47:14.746091 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4469ab94-0efc-4278-ab24-cc7892d1e678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:47:14.767203 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-4469ab94-0efc-4278-ab24-cc7892d1e678 None None] MFA Rules not processed for user `026744699ed54dce8dc240e44e8aefb3`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:47:14.768378 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-4469ab94-0efc-4278-ab24-cc7892d1e678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:47:14.827237 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1157/2333] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:47:14 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:47:17.889543 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=31 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:47:21.271933 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c28085-0d03-4dfe-8fa1-c07f01697fdd None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:47:21.274226 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c28085-0d03-4dfe-8fa1-c07f01697fdd None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:47:21.274744 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c28085-0d03-4dfe-8fa1-c07f01697fdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:47:21.308621 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-99c28085-0d03-4dfe-8fa1-c07f01697fdd None None] MFA Rules not processed for user `52f796d5013d4a31b107fb86c5523c9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:47:21.310221 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-99c28085-0d03-4dfe-8fa1-c07f01697fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:47:21.318016 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1177/2334] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:47:21 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:47:22.058831 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4469ab94-0efc-4278-ab24-cc7892d1e678 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:47:22.080866 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4469ab94-0efc-4278-ab24-cc7892d1e678 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:47:22.087671 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459714a4-5db2-4271-be71-99488a03b694 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:47:22.088880 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459714a4-5db2-4271-be71-99488a03b694 service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:47:22.089396 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459714a4-5db2-4271-be71-99488a03b694 service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:47:22.089882 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459714a4-5db2-4271-be71-99488a03b694 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:47:22.100183 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-459714a4-5db2-4271-be71-99488a03b694 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:47:22.108620 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459714a4-5db2-4271-be71-99488a03b694 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:47:22.110933 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459714a4-5db2-4271-be71-99488a03b694 service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:47:22.121593 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1158/2335] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:47:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:47:22.552217 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c28085-0d03-4dfe-8fa1-c07f01697fdd None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:47:22.554589 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-99c28085-0d03-4dfe-8fa1-c07f01697fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:47:22.572281 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c28085-0d03-4dfe-8fa1-c07f01697fdd None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:47:22.580063 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70c23cf-5dff-480d-9228-62425fe9fa77 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4f49c1b08c460f8ba6c451c9c92c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:47:22.580063 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70c23cf-5dff-480d-9228-62425fe9fa77 service nova] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:47:22.580063 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70c23cf-5dff-480d-9228-62425fe9fa77 service nova] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:47:22.580453 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70c23cf-5dff-480d-9228-62425fe9fa77 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:47:22.582352 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-a70c23cf-5dff-480d-9228-62425fe9fa77 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:47:22.597445 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70c23cf-5dff-480d-9228-62425fe9fa77 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:47:22.599710 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70c23cf-5dff-480d-9228-62425fe9fa77 service nova] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:47:22.615495 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1178/2336] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:47:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:47:26.522980 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459714a4-5db2-4271-be71-99488a03b694 service neutron] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:47:26.532123 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=67 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:47:26.538460 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459714a4-5db2-4271-be71-99488a03b694 service neutron] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:47:26.546039 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c230b64-5b3d-462b-8248-08eccae5b072 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4f49c1b08c460f8ba6c451c9c92c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:47:26.552052 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c230b64-5b3d-462b-8248-08eccae5b072 service nova] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:47:26.552439 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c230b64-5b3d-462b-8248-08eccae5b072 service nova] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:47:26.552817 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c230b64-5b3d-462b-8248-08eccae5b072 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:47:26.555255 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-4c230b64-5b3d-462b-8248-08eccae5b072 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:47:26.580089 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c230b64-5b3d-462b-8248-08eccae5b072 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:47:26.582881 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c230b64-5b3d-462b-8248-08eccae5b072 service nova] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:47:26.594416 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1159/2337] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:47:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:47:32.609599 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=9 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:47:34.823277 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdee808-5817-44f0-85d3-28a1afedf8ee None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:47:34.823964 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdee808-5817-44f0-85d3-28a1afedf8ee None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:47:34.823964 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdee808-5817-44f0-85d3-28a1afedf8ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:47:34.848678 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-dcdee808-5817-44f0-85d3-28a1afedf8ee None None] MFA Rules not processed for user `ea70acd744994250a2905b5dd7163dfb`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:47:34.851755 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-dcdee808-5817-44f0-85d3-28a1afedf8ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:47:34.865052 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1179/2338] 10.222.0.47 () {58 vars in 967 bytes} [Wed Aug 24 06:47:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:47:35.089699 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c230b64-5b3d-462b-8248-08eccae5b072 service nova] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:47:35.102711 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c230b64-5b3d-462b-8248-08eccae5b072 service nova] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:47:35.108731 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bcadbd-3b5d-42e9-8ff3-4a96e28d083b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f1cbaf2cd6427585f298c9d58c319a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:47:35.109503 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bcadbd-3b5d-42e9-8ff3-4a96e28d083b service swift] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:47:35.109910 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bcadbd-3b5d-42e9-8ff3-4a96e28d083b service swift] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:47:35.110287 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bcadbd-3b5d-42e9-8ff3-4a96e28d083b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:47:35.120860 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bcadbd-3b5d-42e9-8ff3-4a96e28d083b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:47:35.122778 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bcadbd-3b5d-42e9-8ff3-4a96e28d083b service swift] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:47:35.132642 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1160/2339] 10.222.0.47 () {60 vars in 1384 bytes} [Wed Aug 24 06:47:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 06:47:41.286902 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa684be-a5bc-4a64-b8f3-3376e645abb5 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:47:41.287433 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa684be-a5bc-4a64-b8f3-3376e645abb5 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:47:41.287550 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa684be-a5bc-4a64-b8f3-3376e645abb5 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:47:41.289575 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1180/2340] 10.222.0.47 () {54 vars in 921 bytes} [Wed Aug 24 06:47:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 06:47:41.303385 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee2605-ed74-4393-a898-83e2593c6956 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:47:41.303963 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee2605-ed74-4393-a898-83e2593c6956 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:47:41.304181 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee2605-ed74-4393-a898-83e2593c6956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:47:41.330694 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-2eee2605-ed74-4393-a898-83e2593c6956 None None] MFA Rules not processed for user `306188d42dbd4b5b84082ab3842eaf9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:47:41.332128 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-2eee2605-ed74-4393-a898-83e2593c6956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:47:41.342502 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1161/2341] 10.222.0.47 () {58 vars in 1018 bytes} [Wed Aug 24 06:47:41 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:47:41.358376 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa684be-a5bc-4a64-b8f3-3376e645abb5 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:47:41.374989 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa684be-a5bc-4a64-b8f3-3376e645abb5 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:47:41.380310 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b6b530-48a1-4231-a00e-934cf4bfde37 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '306188d42dbd4b5b84082ab3842eaf9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:47:41.383919 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b6b530-48a1-4231-a00e-934cf4bfde37 service placement] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:47:41.383919 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b6b530-48a1-4231-a00e-934cf4bfde37 service placement] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:47:41.383919 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b6b530-48a1-4231-a00e-934cf4bfde37 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:47:41.394183 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b6b530-48a1-4231-a00e-934cf4bfde37 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:47:41.395911 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b6b530-48a1-4231-a00e-934cf4bfde37 service placement] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:47:41.407414 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1181/2342] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:47:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:47:46.190945 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eee2605-ed74-4393-a898-83e2593c6956 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:47:46.215456 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eee2605-ed74-4393-a898-83e2593c6956 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:47:46.215456 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c43af7-f660-415a-b894-7a9fcddbb4bb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '306188d42dbd4b5b84082ab3842eaf9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:47:46.215456 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c43af7-f660-415a-b894-7a9fcddbb4bb service placement] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:47:46.215456 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c43af7-f660-415a-b894-7a9fcddbb4bb service placement] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:47:46.216007 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c43af7-f660-415a-b894-7a9fcddbb4bb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:47:46.219321 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-72c43af7-f660-415a-b894-7a9fcddbb4bb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:47:46.238056 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c43af7-f660-415a-b894-7a9fcddbb4bb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:47:46.240904 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c43af7-f660-415a-b894-7a9fcddbb4bb service placement] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:47:46.253018 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1162/2343] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:47:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:47:46.388130 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b6b530-48a1-4231-a00e-934cf4bfde37 service placement] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:47:46.390541 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-59b6b530-48a1-4231-a00e-934cf4bfde37 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:47:46.509131 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b6b530-48a1-4231-a00e-934cf4bfde37 service placement] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:47:46.518085 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0cc96d-20c3-4880-812b-61797b1e72a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:47:46.518872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0cc96d-20c3-4880-812b-61797b1e72a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:47:46.519224 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0cc96d-20c3-4880-812b-61797b1e72a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:47:46.519569 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0cc96d-20c3-4880-812b-61797b1e72a2 admin admin] PATH_INFO: `/v3/users/fc1fefc915e941628a01336564368f81` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:47:46.521120 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0cc96d-20c3-4880-812b-61797b1e72a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc1fefc915e941628a01336564368f81)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:47:46.523158 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0cc96d-20c3-4880-812b-61797b1e72a2 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:47:46.785801 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-1e0cc96d-20c3-4880-812b-61797b1e72a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc1fefc915e941628a01336564368f81'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:47:46.786473 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-1e0cc96d-20c3-4880-812b-61797b1e72a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc1fefc915e941628a01336564368f81'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:47:46.787087 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-1e0cc96d-20c3-4880-812b-61797b1e72a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc1fefc915e941628a01336564368f81'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:47:46.894754 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-1e0cc96d-20c3-4880-812b-61797b1e72a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc1fefc915e941628a01336564368f81'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:47:46.909822 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-1e0cc96d-20c3-4880-812b-61797b1e72a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc1fefc915e941628a01336564368f81'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:47:46.924228 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1182/2344] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:47:46 2022] DELETE /identity/v3/users/fc1fefc915e941628a01336564368f81 => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:47:47.210776 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c43af7-f660-415a-b894-7a9fcddbb4bb service placement] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:47:47.232782 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c43af7-f660-415a-b894-7a9fcddbb4bb service placement] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:47:47.291767 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f3a95d-621f-4766-84ca-a80cf2cb3e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:47:47.292335 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f3a95d-621f-4766-84ca-a80cf2cb3e38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:47:47.292616 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f3a95d-621f-4766-84ca-a80cf2cb3e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:47:47.292944 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f3a95d-621f-4766-84ca-a80cf2cb3e38 admin admin] PATH_INFO: `/v3/projects/e36bdfb532924d8ebe001112e285fcdc` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:47:47.294101 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f3a95d-621f-4766-84ca-a80cf2cb3e38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e36bdfb532924d8ebe001112e285fcdc)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:47:47.295991 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f3a95d-621f-4766-84ca-a80cf2cb3e38 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:47:47.415299 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-e3f3a95d-621f-4766-84ca-a80cf2cb3e38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e36bdfb532924d8ebe001112e285fcdc', 'request_id': 'req-e3f3a95d-621f-4766-84ca-a80cf2cb3e38'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:47:47.420984 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-e3f3a95d-621f-4766-84ca-a80cf2cb3e38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e36bdfb532924d8ebe001112e285fcdc', 'request_id': 'req-e3f3a95d-621f-4766-84ca-a80cf2cb3e38'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:47:47.495049 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-e3f3a95d-621f-4766-84ca-a80cf2cb3e38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e36bdfb532924d8ebe001112e285fcdc', 'request_id': 'req-e3f3a95d-621f-4766-84ca-a80cf2cb3e38'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:47:47.506816 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-e3f3a95d-621f-4766-84ca-a80cf2cb3e38 admin admin] The token cache is being invalidate because project e36bdfb532924d8ebe001112e285fcdc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:47:47.507108 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-e3f3a95d-621f-4766-84ca-a80cf2cb3e38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:47:47.507750 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-e3f3a95d-621f-4766-84ca-a80cf2cb3e38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:47:47.509316 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1163/2345] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:47:47 2022] DELETE /identity/v3/projects/e36bdfb532924d8ebe001112e285fcdc => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:47:47.514047 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0cc96d-20c3-4880-812b-61797b1e72a2 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:47:47.518813 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-1e0cc96d-20c3-4880-812b-61797b1e72a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:47:47.529593 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0cc96d-20c3-4880-812b-61797b1e72a2 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:47:47.587240 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb71744a-e449-4411-a29a-58cc363fbe3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:47:47.588018 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb71744a-e449-4411-a29a-58cc363fbe3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:47:47.588313 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb71744a-e449-4411-a29a-58cc363fbe3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:47:47.588749 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb71744a-e449-4411-a29a-58cc363fbe3d admin admin] PATH_INFO: `/v3/users/61973fd08a0a418ba32dff8ead839d7a` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:47:47.589772 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb71744a-e449-4411-a29a-58cc363fbe3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=61973fd08a0a418ba32dff8ead839d7a)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:47:47.591710 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb71744a-e449-4411-a29a-58cc363fbe3d admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:47:47.593288 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:47:47.834396 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-eb71744a-e449-4411-a29a-58cc363fbe3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61973fd08a0a418ba32dff8ead839d7a'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:47:47.835077 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-eb71744a-e449-4411-a29a-58cc363fbe3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61973fd08a0a418ba32dff8ead839d7a'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:47:47.835677 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-eb71744a-e449-4411-a29a-58cc363fbe3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61973fd08a0a418ba32dff8ead839d7a'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:47:47.905275 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-eb71744a-e449-4411-a29a-58cc363fbe3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61973fd08a0a418ba32dff8ead839d7a'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:47:47.918077 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-eb71744a-e449-4411-a29a-58cc363fbe3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61973fd08a0a418ba32dff8ead839d7a'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:47:47.930341 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1183/2346] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:47:47 2022] DELETE /identity/v3/users/61973fd08a0a418ba32dff8ead839d7a => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:47:48.268655 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f3a95d-621f-4766-84ca-a80cf2cb3e38 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:47:48.270458 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-e3f3a95d-621f-4766-84ca-a80cf2cb3e38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:47:48.289060 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f3a95d-621f-4766-84ca-a80cf2cb3e38 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:47:48.335207 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:47:48.348254 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de537ab6-ea25-4728-9a45-c3e090300a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:47:48.348892 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de537ab6-ea25-4728-9a45-c3e090300a1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:47:48.349168 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de537ab6-ea25-4728-9a45-c3e090300a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:47:48.349466 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de537ab6-ea25-4728-9a45-c3e090300a1c admin admin] PATH_INFO: `/v3/projects/b1db1077e83d4ca89034cd358ef15644` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:47:48.350611 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de537ab6-ea25-4728-9a45-c3e090300a1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1db1077e83d4ca89034cd358ef15644)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:47:48.352546 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de537ab6-ea25-4728-9a45-c3e090300a1c admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:47:48.462056 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-de537ab6-ea25-4728-9a45-c3e090300a1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1db1077e83d4ca89034cd358ef15644', 'request_id': 'req-de537ab6-ea25-4728-9a45-c3e090300a1c'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:47:48.467523 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-de537ab6-ea25-4728-9a45-c3e090300a1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1db1077e83d4ca89034cd358ef15644', 'request_id': 'req-de537ab6-ea25-4728-9a45-c3e090300a1c'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:47:48.529167 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-de537ab6-ea25-4728-9a45-c3e090300a1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1db1077e83d4ca89034cd358ef15644', 'request_id': 'req-de537ab6-ea25-4728-9a45-c3e090300a1c'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:47:48.541671 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-de537ab6-ea25-4728-9a45-c3e090300a1c admin admin] The token cache is being invalidate because project b1db1077e83d4ca89034cd358ef15644 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:47:48.541964 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-de537ab6-ea25-4728-9a45-c3e090300a1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:47:48.542626 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-de537ab6-ea25-4728-9a45-c3e090300a1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:47:48.544348 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1164/2347] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:47:48 2022] DELETE /identity/v3/projects/b1db1077e83d4ca89034cd358ef15644 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:47:57.926823 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=26,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:47:58.537884 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=30,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:48:04.926552 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8618a8b9-8dd5-4159-86ac-9fc74f439b5a None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:48:04.927155 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8618a8b9-8dd5-4159-86ac-9fc74f439b5a None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:48:04.927155 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8618a8b9-8dd5-4159-86ac-9fc74f439b5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:48:04.946978 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-8618a8b9-8dd5-4159-86ac-9fc74f439b5a None None] MFA Rules not processed for user `8b9b29b53a7b490da29fbf6e5451c362`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:48:04.948681 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-8618a8b9-8dd5-4159-86ac-9fc74f439b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:48:05.017318 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1184/2348] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:48:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:48:15.007806 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=25 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:48:26.791257 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de537ab6-ea25-4728-9a45-c3e090300a1c admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:48:26.792679 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-de537ab6-ea25-4728-9a45-c3e090300a1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:48:26.850210 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de537ab6-ea25-4728-9a45-c3e090300a1c admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:48:26.853782 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7182a01c-7501-4eb9-8d17-00fcfd3f8f7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36639dce4f97412f89d71367f650b901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:48:26.854394 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7182a01c-7501-4eb9-8d17-00fcfd3f8f7c service cinder] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:48:26.854670 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7182a01c-7501-4eb9-8d17-00fcfd3f8f7c service cinder] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:48:26.854941 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7182a01c-7501-4eb9-8d17-00fcfd3f8f7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:48:26.856651 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-7182a01c-7501-4eb9-8d17-00fcfd3f8f7c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:48:26.916740 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7182a01c-7501-4eb9-8d17-00fcfd3f8f7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:48:26.918211 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7182a01c-7501-4eb9-8d17-00fcfd3f8f7c service cinder] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:48:26.928168 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1165/2349] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:48:26 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:48:36.924071 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=48 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:48:47.499057 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8618a8b9-8dd5-4159-86ac-9fc74f439b5a None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:48:47.500615 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-8618a8b9-8dd5-4159-86ac-9fc74f439b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:48:47.667855 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8618a8b9-8dd5-4159-86ac-9fc74f439b5a None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:48:47.676008 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596e12e0-0027-4cc2-ac8f-8a1642bc87c7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '306188d42dbd4b5b84082ab3842eaf9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:48:47.677015 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596e12e0-0027-4cc2-ac8f-8a1642bc87c7 service placement] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:48:47.677551 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596e12e0-0027-4cc2-ac8f-8a1642bc87c7 service placement] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:48:47.678031 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596e12e0-0027-4cc2-ac8f-8a1642bc87c7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:48:47.680843 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-596e12e0-0027-4cc2-ac8f-8a1642bc87c7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:48:47.701437 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596e12e0-0027-4cc2-ac8f-8a1642bc87c7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:48:47.703164 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596e12e0-0027-4cc2-ac8f-8a1642bc87c7 service placement] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:48:47.715806 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1185/2350] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:48:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:48:48.587354 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7182a01c-7501-4eb9-8d17-00fcfd3f8f7c service cinder] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:48:48.588784 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-7182a01c-7501-4eb9-8d17-00fcfd3f8f7c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:48:48.667831 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7182a01c-7501-4eb9-8d17-00fcfd3f8f7c service cinder] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:48:48.671503 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c9ca6e-368a-44fa-a08d-5dd435b8758e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:48:48.672109 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c9ca6e-368a-44fa-a08d-5dd435b8758e service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:48:48.672389 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c9ca6e-368a-44fa-a08d-5dd435b8758e service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:48:48.672805 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c9ca6e-368a-44fa-a08d-5dd435b8758e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:48:48.675003 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-28c9ca6e-368a-44fa-a08d-5dd435b8758e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:48:48.760795 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c9ca6e-368a-44fa-a08d-5dd435b8758e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:48:48.762272 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c9ca6e-368a-44fa-a08d-5dd435b8758e service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:48:48.782670 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1166/2351] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:48:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:48:52.627530 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596e12e0-0027-4cc2-ac8f-8a1642bc87c7 service placement] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:48:52.629089 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-596e12e0-0027-4cc2-ac8f-8a1642bc87c7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:48:52.649966 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596e12e0-0027-4cc2-ac8f-8a1642bc87c7 service placement] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:48:52.654942 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a892f273-6bf5-4ea0-99e2-2d7444dd6eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:48:52.655639 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a892f273-6bf5-4ea0-99e2-2d7444dd6eb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:48:52.655931 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a892f273-6bf5-4ea0-99e2-2d7444dd6eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:48:52.656212 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a892f273-6bf5-4ea0-99e2-2d7444dd6eb6 admin admin] PATH_INFO: `/v3/users/026744699ed54dce8dc240e44e8aefb3` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:48:52.658079 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a892f273-6bf5-4ea0-99e2-2d7444dd6eb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=026744699ed54dce8dc240e44e8aefb3)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:48:52.659982 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a892f273-6bf5-4ea0-99e2-2d7444dd6eb6 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:48:52.862258 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-a892f273-6bf5-4ea0-99e2-2d7444dd6eb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '026744699ed54dce8dc240e44e8aefb3'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:48:52.863479 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-a892f273-6bf5-4ea0-99e2-2d7444dd6eb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '026744699ed54dce8dc240e44e8aefb3'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:48:52.864143 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-a892f273-6bf5-4ea0-99e2-2d7444dd6eb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '026744699ed54dce8dc240e44e8aefb3'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:48:52.922773 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-a892f273-6bf5-4ea0-99e2-2d7444dd6eb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '026744699ed54dce8dc240e44e8aefb3'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:48:52.939371 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-a892f273-6bf5-4ea0-99e2-2d7444dd6eb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '026744699ed54dce8dc240e44e8aefb3'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:48:52.946296 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:48:52.955977 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1186/2352] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:48:52 2022] DELETE /identity/v3/users/026744699ed54dce8dc240e44e8aefb3 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:48:53.199088 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c9ca6e-368a-44fa-a08d-5dd435b8758e service neutron] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:48:53.209650 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c9ca6e-368a-44fa-a08d-5dd435b8758e service neutron] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:48:53.281857 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddffa9e-d623-404a-abed-480708173d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:48:53.282510 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddffa9e-d623-404a-abed-480708173d7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:48:53.282786 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddffa9e-d623-404a-abed-480708173d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:48:53.283133 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddffa9e-d623-404a-abed-480708173d7c admin admin] PATH_INFO: `/v3/projects/50896ec327454f28b1987cd4271b6e27` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:48:53.284179 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddffa9e-d623-404a-abed-480708173d7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=50896ec327454f28b1987cd4271b6e27)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:48:53.286225 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddffa9e-d623-404a-abed-480708173d7c admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:48:53.288574 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:48:53.430392 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-eddffa9e-d623-404a-abed-480708173d7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50896ec327454f28b1987cd4271b6e27', 'request_id': 'req-eddffa9e-d623-404a-abed-480708173d7c'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:48:53.437879 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-eddffa9e-d623-404a-abed-480708173d7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50896ec327454f28b1987cd4271b6e27', 'request_id': 'req-eddffa9e-d623-404a-abed-480708173d7c'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:48:53.505915 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-eddffa9e-d623-404a-abed-480708173d7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50896ec327454f28b1987cd4271b6e27', 'request_id': 'req-eddffa9e-d623-404a-abed-480708173d7c'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:48:53.518951 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-eddffa9e-d623-404a-abed-480708173d7c admin admin] The token cache is being invalidate because project 50896ec327454f28b1987cd4271b6e27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:48:53.519213 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-eddffa9e-d623-404a-abed-480708173d7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:48:53.519877 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-eddffa9e-d623-404a-abed-480708173d7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:48:53.522552 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1167/2353] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:48:53 2022] DELETE /identity/v3/projects/50896ec327454f28b1987cd4271b6e27 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:48:53.528323 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1024e3-391a-44a5-985b-a9ee0cbe5428 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:48:53.528808 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1024e3-391a-44a5-985b-a9ee0cbe5428 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:48:53.529340 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1024e3-391a-44a5-985b-a9ee0cbe5428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:48:53.596659 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-aa1024e3-391a-44a5-985b-a9ee0cbe5428 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:48:53.598838 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-aa1024e3-391a-44a5-985b-a9ee0cbe5428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:48:53.667201 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1187/2354] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:48:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:48:53.675334 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899dea32-0502-4f48-8c2c-a83da9f75751 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:48:53.675620 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899dea32-0502-4f48-8c2c-a83da9f75751 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:48:53.675901 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899dea32-0502-4f48-8c2c-a83da9f75751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:48:53.699427 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-899dea32-0502-4f48-8c2c-a83da9f75751 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:48:53.700662 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-899dea32-0502-4f48-8c2c-a83da9f75751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:48:53.709642 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1168/2355] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:48:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:48:53.714640 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1024e3-391a-44a5-985b-a9ee0cbe5428 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:48:53.725473 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1024e3-391a-44a5-985b-a9ee0cbe5428 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:48:53.732224 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6920ca-c006-4b6b-87cd-2f5b5049c6e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:48:53.732831 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6920ca-c006-4b6b-87cd-2f5b5049c6e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:48:53.733111 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6920ca-c006-4b6b-87cd-2f5b5049c6e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:48:53.733423 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6920ca-c006-4b6b-87cd-2f5b5049c6e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:48:53.734074 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6920ca-c006-4b6b-87cd-2f5b5049c6e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:48:53.734378 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6920ca-c006-4b6b-87cd-2f5b5049c6e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:48:53.736434 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6920ca-c006-4b6b-87cd-2f5b5049c6e7 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:48:53.749098 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1188/2356] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 24 06:48:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:48:53.754320 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899dea32-0502-4f48-8c2c-a83da9f75751 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:48:53.763856 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899dea32-0502-4f48-8c2c-a83da9f75751 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:48:53.769815 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9275f3-d93d-4482-935c-480eb046156c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:48:53.770424 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9275f3-d93d-4482-935c-480eb046156c admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:48:53.770706 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9275f3-d93d-4482-935c-480eb046156c admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:48:53.770979 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9275f3-d93d-4482-935c-480eb046156c admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:48:53.771610 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9275f3-d93d-4482-935c-480eb046156c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:48:53.773556 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9275f3-d93d-4482-935c-480eb046156c admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:48:53.867044 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1169/2357] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:48:53 2022] POST /identity/v3/projects => generated 367 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:48:53.872762 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6920ca-c006-4b6b-87cd-2f5b5049c6e7 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:48:53.883751 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6920ca-c006-4b6b-87cd-2f5b5049c6e7 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:48:53.946049 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac1c80e-714c-4e8d-aef6-8118269f29d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:48:53.946661 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac1c80e-714c-4e8d-aef6-8118269f29d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:48:53.946947 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac1c80e-714c-4e8d-aef6-8118269f29d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:48:53.947220 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac1c80e-714c-4e8d-aef6-8118269f29d3 admin admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:48:53.947857 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac1c80e-714c-4e8d-aef6-8118269f29d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:48:53.949815 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac1c80e-714c-4e8d-aef6-8118269f29d3 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:48:54.020294 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1189/2358] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:48:53 2022] POST /identity/v3/users => generated 284 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:48:54.024439 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9275f3-d93d-4482-935c-480eb046156c admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:48:54.036038 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9275f3-d93d-4482-935c-480eb046156c admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:48:54.045258 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd8077d-f0eb-4d26-93cf-02019f177044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:48:54.045875 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd8077d-f0eb-4d26-93cf-02019f177044 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:48:54.046158 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd8077d-f0eb-4d26-93cf-02019f177044 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:48:54.046463 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd8077d-f0eb-4d26-93cf-02019f177044 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:48:54.047119 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd8077d-f0eb-4d26-93cf-02019f177044 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:48:54.049934 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd8077d-f0eb-4d26-93cf-02019f177044 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:48:54.060540 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1170/2359] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:48:54 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:48:54.065894 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac1c80e-714c-4e8d-aef6-8118269f29d3 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:48:54.076475 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac1c80e-714c-4e8d-aef6-8118269f29d3 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:48:54.081448 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97f4903-8822-46a6-8264-490931cbec30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:48:54.082062 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97f4903-8822-46a6-8264-490931cbec30 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:48:54.082348 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97f4903-8822-46a6-8264-490931cbec30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:48:54.082628 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97f4903-8822-46a6-8264-490931cbec30 admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:48:54.083278 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97f4903-8822-46a6-8264-490931cbec30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:48:54.084887 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97f4903-8822-46a6-8264-490931cbec30 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:48:54.096404 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1190/2360] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:48:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:48:54.108306 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd8077d-f0eb-4d26-93cf-02019f177044 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:48:54.120129 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd8077d-f0eb-4d26-93cf-02019f177044 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:48:54.124810 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268e341f-a031-47a7-9ca3-1fe362746aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:48:54.126268 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268e341f-a031-47a7-9ca3-1fe362746aeb admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:48:54.126673 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268e341f-a031-47a7-9ca3-1fe362746aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:48:54.126958 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268e341f-a031-47a7-9ca3-1fe362746aeb admin admin] PATH_INFO: `/v3/projects/02539f920e94406396ad41583ab944c3/users/af553747631b474e9c28679563b3dd42/roles/e1f9b167c51c4a388cbc9594db8a465d` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:48:54.142085 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268e341f-a031-47a7-9ca3-1fe362746aeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=02539f920e94406396ad41583ab944c3, user_id=af553747631b474e9c28679563b3dd42, role_id=e1f9b167c51c4a388cbc9594db8a465d)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:48:54.144746 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268e341f-a031-47a7-9ca3-1fe362746aeb admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:48:54.254779 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1171/2361] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:48:54 2022] PUT /identity/v3/projects/02539f920e94406396ad41583ab944c3/users/af553747631b474e9c28679563b3dd42/roles/e1f9b167c51c4a388cbc9594db8a465d => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:48:54.265247 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3499d2ce-6b53-4684-970a-b7a53ece1db6 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:48:54.265533 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3499d2ce-6b53-4684-970a-b7a53ece1db6 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:48:54.265804 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3499d2ce-6b53-4684-970a-b7a53ece1db6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:48:54.293721 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-3499d2ce-6b53-4684-970a-b7a53ece1db6 None None] MFA Rules not processed for user `af553747631b474e9c28679563b3dd42`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:48:54.294845 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-3499d2ce-6b53-4684-970a-b7a53ece1db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:48:54.312508 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:48:54.364160 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1191/2362] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:48:54 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:48:54.375334 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268e341f-a031-47a7-9ca3-1fe362746aeb admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:48:54.377549 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-268e341f-a031-47a7-9ca3-1fe362746aeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:48:54.435118 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268e341f-a031-47a7-9ca3-1fe362746aeb admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:48:54.439436 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8a0f0a-5258-48e3-8e30-4fe9b6bd5f17 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36639dce4f97412f89d71367f650b901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:48:54.440099 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8a0f0a-5258-48e3-8e30-4fe9b6bd5f17 service cinder] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:48:54.440387 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8a0f0a-5258-48e3-8e30-4fe9b6bd5f17 service cinder] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:48:54.440670 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8a0f0a-5258-48e3-8e30-4fe9b6bd5f17 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:48:54.449284 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8a0f0a-5258-48e3-8e30-4fe9b6bd5f17 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:48:54.450717 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8a0f0a-5258-48e3-8e30-4fe9b6bd5f17 service cinder] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:48:54.460164 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1172/2363] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:48:54 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:49:04.408249 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=19 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:49:04.456061 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=76,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:49:04.460166 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3499d2ce-6b53-4684-970a-b7a53ece1db6 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:04.522729 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3499d2ce-6b53-4684-970a-b7a53ece1db6 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:04.527312 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93318d3c-7a6c-437e-bce7-1670c12053e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:04.527933 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93318d3c-7a6c-437e-bce7-1670c12053e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:04.528211 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93318d3c-7a6c-437e-bce7-1670c12053e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:04.528478 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93318d3c-7a6c-437e-bce7-1670c12053e7 admin admin] PATH_INFO: `/v3/users/af553747631b474e9c28679563b3dd42` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:04.529610 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93318d3c-7a6c-437e-bce7-1670c12053e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af553747631b474e9c28679563b3dd42)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:04.531542 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93318d3c-7a6c-437e-bce7-1670c12053e7 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:04.737171 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-93318d3c-7a6c-437e-bce7-1670c12053e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af553747631b474e9c28679563b3dd42'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:04.737863 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-93318d3c-7a6c-437e-bce7-1670c12053e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af553747631b474e9c28679563b3dd42'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:04.738480 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-93318d3c-7a6c-437e-bce7-1670c12053e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af553747631b474e9c28679563b3dd42'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:04.813426 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-93318d3c-7a6c-437e-bce7-1670c12053e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af553747631b474e9c28679563b3dd42'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:04.827290 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-93318d3c-7a6c-437e-bce7-1670c12053e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af553747631b474e9c28679563b3dd42'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:04.840408 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1192/2364] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:49:04 2022] DELETE /identity/v3/users/af553747631b474e9c28679563b3dd42 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:49:04.845438 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daf2845-8545-4daf-9d4c-7b6bfdc96fa0 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:04.845726 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daf2845-8545-4daf-9d4c-7b6bfdc96fa0 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:04.846005 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daf2845-8545-4daf-9d4c-7b6bfdc96fa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:04.869539 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-8daf2845-8545-4daf-9d4c-7b6bfdc96fa0 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:49:04.870783 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-8daf2845-8545-4daf-9d4c-7b6bfdc96fa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:49:04.930410 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1173/2365] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:49:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:49:04.940106 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93318d3c-7a6c-437e-bce7-1670c12053e7 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:04.941639 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-93318d3c-7a6c-437e-bce7-1670c12053e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:49:05.022447 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93318d3c-7a6c-437e-bce7-1670c12053e7 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:05.026556 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28286a9d-971f-4051-9ff4-943e769d6306 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:05.027371 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28286a9d-971f-4051-9ff4-943e769d6306 service neutron] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:05.027371 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28286a9d-971f-4051-9ff4-943e769d6306 service neutron] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:05.028368 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28286a9d-971f-4051-9ff4-943e769d6306 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:05.038159 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28286a9d-971f-4051-9ff4-943e769d6306 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:05.039885 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28286a9d-971f-4051-9ff4-943e769d6306 service neutron] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:05.049313 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1193/2366] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:49:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:49:05.538251 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daf2845-8545-4daf-9d4c-7b6bfdc96fa0 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:05.540144 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-8daf2845-8545-4daf-9d4c-7b6bfdc96fa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:49:05.552351 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daf2845-8545-4daf-9d4c-7b6bfdc96fa0 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:05.558683 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e2e555-feba-4c4d-af76-d9aeebf2ec50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:05.559387 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e2e555-feba-4c4d-af76-d9aeebf2ec50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:05.559665 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e2e555-feba-4c4d-af76-d9aeebf2ec50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:05.559930 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e2e555-feba-4c4d-af76-d9aeebf2ec50 admin admin] PATH_INFO: `/v3/projects/02539f920e94406396ad41583ab944c3` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:05.561277 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e2e555-feba-4c4d-af76-d9aeebf2ec50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02539f920e94406396ad41583ab944c3)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:05.563192 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e2e555-feba-4c4d-af76-d9aeebf2ec50 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:05.655853 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-d7e2e555-feba-4c4d-af76-d9aeebf2ec50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02539f920e94406396ad41583ab944c3', 'request_id': 'req-d7e2e555-feba-4c4d-af76-d9aeebf2ec50'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:05.662263 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-d7e2e555-feba-4c4d-af76-d9aeebf2ec50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02539f920e94406396ad41583ab944c3', 'request_id': 'req-d7e2e555-feba-4c4d-af76-d9aeebf2ec50'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:05.738915 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-d7e2e555-feba-4c4d-af76-d9aeebf2ec50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02539f920e94406396ad41583ab944c3', 'request_id': 'req-d7e2e555-feba-4c4d-af76-d9aeebf2ec50'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:05.751655 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-d7e2e555-feba-4c4d-af76-d9aeebf2ec50 admin admin] The token cache is being invalidate because project 02539f920e94406396ad41583ab944c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:49:05.751942 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-d7e2e555-feba-4c4d-af76-d9aeebf2ec50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:05.752711 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-d7e2e555-feba-4c4d-af76-d9aeebf2ec50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:49:05.754270 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1174/2367] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:49:05 2022] DELETE /identity/v3/projects/02539f920e94406396ad41583ab944c3 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:49:05.760960 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3da9c0-b680-4ac8-bbd5-2cf6854e8647 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:05.761503 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3da9c0-b680-4ac8-bbd5-2cf6854e8647 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:05.761853 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3da9c0-b680-4ac8-bbd5-2cf6854e8647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:05.769464 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:49:05.783579 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-4b3da9c0-b680-4ac8-bbd5-2cf6854e8647 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:49:05.784786 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-4b3da9c0-b680-4ac8-bbd5-2cf6854e8647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:49:05.846366 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1194/2368] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:49:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:49:05.855602 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12392966-763e-4f10-aa49-674eac4de695 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:05.855889 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12392966-763e-4f10-aa49-674eac4de695 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:05.856169 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12392966-763e-4f10-aa49-674eac4de695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:05.876842 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-12392966-763e-4f10-aa49-674eac4de695 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:49:05.878112 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-12392966-763e-4f10-aa49-674eac4de695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:49:05.886965 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1175/2369] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:49:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:49:05.890909 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3da9c0-b680-4ac8-bbd5-2cf6854e8647 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:05.901810 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3da9c0-b680-4ac8-bbd5-2cf6854e8647 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:05.906626 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8626a79-776b-4d9f-ad1e-f54e69c41ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:05.907233 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8626a79-776b-4d9f-ad1e-f54e69c41ef4 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:05.907516 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8626a79-776b-4d9f-ad1e-f54e69c41ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:05.907811 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8626a79-776b-4d9f-ad1e-f54e69c41ef4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:05.908456 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8626a79-776b-4d9f-ad1e-f54e69c41ef4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:49:05.908826 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8626a79-776b-4d9f-ad1e-f54e69c41ef4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:05.910499 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8626a79-776b-4d9f-ad1e-f54e69c41ef4 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:05.923263 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1195/2370] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 24 06:49:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:49:05.927672 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12392966-763e-4f10-aa49-674eac4de695 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:05.938481 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12392966-763e-4f10-aa49-674eac4de695 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:05.943047 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee80395a-bbc3-4603-8145-039d65282442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:05.943639 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee80395a-bbc3-4603-8145-039d65282442 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:05.943914 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee80395a-bbc3-4603-8145-039d65282442 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:05.944180 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee80395a-bbc3-4603-8145-039d65282442 admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:05.944816 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee80395a-bbc3-4603-8145-039d65282442 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:05.946844 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee80395a-bbc3-4603-8145-039d65282442 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:06.067193 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1176/2371] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:49:05 2022] POST /identity/v3/projects => generated 371 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:49:06.071907 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8626a79-776b-4d9f-ad1e-f54e69c41ef4 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:06.084934 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8626a79-776b-4d9f-ad1e-f54e69c41ef4 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:06.145556 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59de674-dfc7-4e6c-904d-73c5f149f126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:06.146163 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59de674-dfc7-4e6c-904d-73c5f149f126 admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:06.146438 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59de674-dfc7-4e6c-904d-73c5f149f126 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:06.146861 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59de674-dfc7-4e6c-904d-73c5f149f126 admin admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:06.147326 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59de674-dfc7-4e6c-904d-73c5f149f126 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:06.150129 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59de674-dfc7-4e6c-904d-73c5f149f126 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:06.229167 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1196/2372] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:49:06 2022] POST /identity/v3/users => generated 286 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:49:06.230461 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee80395a-bbc3-4603-8145-039d65282442 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:06.241202 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee80395a-bbc3-4603-8145-039d65282442 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:06.246230 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4172a1-74fa-4eae-82db-a24193ad87d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:06.246845 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4172a1-74fa-4eae-82db-a24193ad87d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:06.247128 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4172a1-74fa-4eae-82db-a24193ad87d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:06.247397 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4172a1-74fa-4eae-82db-a24193ad87d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:06.248028 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4172a1-74fa-4eae-82db-a24193ad87d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:06.249677 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4172a1-74fa-4eae-82db-a24193ad87d9 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:06.266282 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1177/2373] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:49:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:49:06.268888 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59de674-dfc7-4e6c-904d-73c5f149f126 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:06.280085 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59de674-dfc7-4e6c-904d-73c5f149f126 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:06.284988 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a0398c-6789-4eb7-928c-db7abcf26eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:06.285762 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a0398c-6789-4eb7-928c-db7abcf26eab admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:06.286054 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a0398c-6789-4eb7-928c-db7abcf26eab admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:06.286337 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a0398c-6789-4eb7-928c-db7abcf26eab admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:06.287010 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a0398c-6789-4eb7-928c-db7abcf26eab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:06.288849 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a0398c-6789-4eb7-928c-db7abcf26eab admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:06.300246 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1197/2374] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:49:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:49:06.304582 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4172a1-74fa-4eae-82db-a24193ad87d9 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:06.315692 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4172a1-74fa-4eae-82db-a24193ad87d9 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:06.321547 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc97a212-cee5-4303-a4e6-a6796ce78816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:06.322209 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc97a212-cee5-4303-a4e6-a6796ce78816 admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:06.322591 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc97a212-cee5-4303-a4e6-a6796ce78816 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:06.322873 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc97a212-cee5-4303-a4e6-a6796ce78816 admin admin] PATH_INFO: `/v3/projects/644d103f13eb45d1a280f51c1ac4b6b4/users/8113e2d3a2d148fb9c391ed9d5130bcb/roles/e1f9b167c51c4a388cbc9594db8a465d` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:06.338720 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc97a212-cee5-4303-a4e6-a6796ce78816 admin admin] RBAC: Authorizing `identity:create_grant(project_id=644d103f13eb45d1a280f51c1ac4b6b4, user_id=8113e2d3a2d148fb9c391ed9d5130bcb, role_id=e1f9b167c51c4a388cbc9594db8a465d)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:06.341453 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc97a212-cee5-4303-a4e6-a6796ce78816 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:06.424806 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1178/2375] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:49:06 2022] PUT /identity/v3/projects/644d103f13eb45d1a280f51c1ac4b6b4/users/8113e2d3a2d148fb9c391ed9d5130bcb/roles/e1f9b167c51c4a388cbc9594db8a465d => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:49:06.436977 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd64d4d-6f05-4745-9c22-138b11a01c85 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:06.437325 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd64d4d-6f05-4745-9c22-138b11a01c85 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:06.437604 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd64d4d-6f05-4745-9c22-138b11a01c85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:06.467547 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-5bd64d4d-6f05-4745-9c22-138b11a01c85 None None] MFA Rules not processed for user `8113e2d3a2d148fb9c391ed9d5130bcb`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:49:06.468792 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-5bd64d4d-6f05-4745-9c22-138b11a01c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:49:06.528611 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:49:06.533839 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1198/2376] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:49:06 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:49:06.538450 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc97a212-cee5-4303-a4e6-a6796ce78816 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:06.560275 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:49:06.601927 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc97a212-cee5-4303-a4e6-a6796ce78816 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:06.606722 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa73279-341e-4c3e-82c4-404ace82dff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:06.607324 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa73279-341e-4c3e-82c4-404ace82dff1 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:06.607603 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa73279-341e-4c3e-82c4-404ace82dff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:06.607872 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa73279-341e-4c3e-82c4-404ace82dff1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:06.608521 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa73279-341e-4c3e-82c4-404ace82dff1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:06.610448 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa73279-341e-4c3e-82c4-404ace82dff1 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:06.727133 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1179/2377] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:49:06 2022] POST /identity/v3/projects => generated 371 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:49:06.728254 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd64d4d-6f05-4745-9c22-138b11a01c85 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:06.739198 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd64d4d-6f05-4745-9c22-138b11a01c85 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:06.798201 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe4529f-a6b2-4328-a086-6fa0014e8c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:06.798942 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe4529f-a6b2-4328-a086-6fa0014e8c23 admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:06.799113 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe4529f-a6b2-4328-a086-6fa0014e8c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:06.799391 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe4529f-a6b2-4328-a086-6fa0014e8c23 admin admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:06.800054 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe4529f-a6b2-4328-a086-6fa0014e8c23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:06.802104 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe4529f-a6b2-4328-a086-6fa0014e8c23 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:06.876976 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1199/2378] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:49:06 2022] POST /identity/v3/users => generated 292 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:49:06.881012 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa73279-341e-4c3e-82c4-404ace82dff1 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:06.892168 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa73279-341e-4c3e-82c4-404ace82dff1 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:06.896885 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c5dd0c-c9d6-493b-a14e-b32b1cb67cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:06.897585 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c5dd0c-c9d6-493b-a14e-b32b1cb67cab admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:06.897899 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c5dd0c-c9d6-493b-a14e-b32b1cb67cab admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:06.898200 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c5dd0c-c9d6-493b-a14e-b32b1cb67cab admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:06.898874 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c5dd0c-c9d6-493b-a14e-b32b1cb67cab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:06.900486 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c5dd0c-c9d6-493b-a14e-b32b1cb67cab admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:06.912530 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1180/2379] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:49:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:49:06.916829 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe4529f-a6b2-4328-a086-6fa0014e8c23 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:06.927072 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe4529f-a6b2-4328-a086-6fa0014e8c23 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:06.932620 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6878f0a3-cfe6-4fb3-8e46-1b4a66b75ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:06.933316 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6878f0a3-cfe6-4fb3-8e46-1b4a66b75ab8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:06.933627 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6878f0a3-cfe6-4fb3-8e46-1b4a66b75ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:06.933905 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6878f0a3-cfe6-4fb3-8e46-1b4a66b75ab8 admin admin] PATH_INFO: `/v3/projects/3df6c7b57ff24b4a9be7130112eb9b3f/users/2325d4e211f44f5da9da53ab362bc31b/roles/486144ad1bcd417e86acdbc1d14327bf` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:06.949540 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6878f0a3-cfe6-4fb3-8e46-1b4a66b75ab8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3df6c7b57ff24b4a9be7130112eb9b3f, user_id=2325d4e211f44f5da9da53ab362bc31b, role_id=486144ad1bcd417e86acdbc1d14327bf)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:06.952216 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6878f0a3-cfe6-4fb3-8e46-1b4a66b75ab8 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:07.013488 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1200/2380] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:49:06 2022] PUT /identity/v3/projects/3df6c7b57ff24b4a9be7130112eb9b3f/users/2325d4e211f44f5da9da53ab362bc31b/roles/486144ad1bcd417e86acdbc1d14327bf => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:49:07.017355 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c5dd0c-c9d6-493b-a14e-b32b1cb67cab admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:07.028115 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c5dd0c-c9d6-493b-a14e-b32b1cb67cab admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:07.087304 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffbfca1-3e26-4af2-8e99-4add0b486789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:07.087921 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffbfca1-3e26-4af2-8e99-4add0b486789 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:07.088206 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffbfca1-3e26-4af2-8e99-4add0b486789 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:07.088482 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffbfca1-3e26-4af2-8e99-4add0b486789 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:07.089123 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffbfca1-3e26-4af2-8e99-4add0b486789 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:07.090758 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffbfca1-3e26-4af2-8e99-4add0b486789 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:07.101972 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1181/2381] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:49:07 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:49:07.106007 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6878f0a3-cfe6-4fb3-8e46-1b4a66b75ab8 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:07.116557 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6878f0a3-cfe6-4fb3-8e46-1b4a66b75ab8 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:07.122090 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ed8d4b-d06c-499b-a436-5c4342459093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:07.128446 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ed8d4b-d06c-499b-a436-5c4342459093 admin admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:07.128446 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ed8d4b-d06c-499b-a436-5c4342459093 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:07.128446 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ed8d4b-d06c-499b-a436-5c4342459093 admin admin] PATH_INFO: `/v3/projects/3df6c7b57ff24b4a9be7130112eb9b3f/users/2325d4e211f44f5da9da53ab362bc31b/roles/486144ad1bcd417e86acdbc1d14327bf` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:07.128446 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ed8d4b-d06c-499b-a436-5c4342459093 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3df6c7b57ff24b4a9be7130112eb9b3f, user_id=2325d4e211f44f5da9da53ab362bc31b, role_id=486144ad1bcd417e86acdbc1d14327bf)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:07.128446 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ed8d4b-d06c-499b-a436-5c4342459093 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:07.140840 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1201/2382] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:49:07 2022] PUT /identity/v3/projects/3df6c7b57ff24b4a9be7130112eb9b3f/users/2325d4e211f44f5da9da53ab362bc31b/roles/486144ad1bcd417e86acdbc1d14327bf => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:49:07.151991 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a9fae8-9d94-4fba-a90c-8c03c18d5566 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:07.152283 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a9fae8-9d94-4fba-a90c-8c03c18d5566 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:07.152564 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a9fae8-9d94-4fba-a90c-8c03c18d5566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:07.187938 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-05a9fae8-9d94-4fba-a90c-8c03c18d5566 None None] MFA Rules not processed for user `2325d4e211f44f5da9da53ab362bc31b`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:49:07.189245 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-05a9fae8-9d94-4fba-a90c-8c03c18d5566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:49:07.257922 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1182/2383] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:49:07 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:49:07.269676 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ed8d4b-d06c-499b-a436-5c4342459093 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:07.271137 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-57ed8d4b-d06c-499b-a436-5c4342459093 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:49:07.351801 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ed8d4b-d06c-499b-a436-5c4342459093 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:07.355720 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02f4b23-76a7-4115-a346-99309ce05acd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36639dce4f97412f89d71367f650b901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:07.356319 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02f4b23-76a7-4115-a346-99309ce05acd service cinder] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:07.356601 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02f4b23-76a7-4115-a346-99309ce05acd service cinder] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:07.356874 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02f4b23-76a7-4115-a346-99309ce05acd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:07.365812 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02f4b23-76a7-4115-a346-99309ce05acd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:07.367635 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02f4b23-76a7-4115-a346-99309ce05acd service cinder] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:07.376901 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1202/2384] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:49:07 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:49:07.583247 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a9fae8-9d94-4fba-a90c-8c03c18d5566 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:07.593476 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a9fae8-9d94-4fba-a90c-8c03c18d5566 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:07.597256 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58d9fa2-d0e8-4248-b0e0-bd849f3cc99c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36639dce4f97412f89d71367f650b901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:07.597864 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58d9fa2-d0e8-4248-b0e0-bd849f3cc99c service cinder] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:07.598145 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58d9fa2-d0e8-4248-b0e0-bd849f3cc99c service cinder] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:07.598416 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58d9fa2-d0e8-4248-b0e0-bd849f3cc99c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:07.610697 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:49:07.656315 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58d9fa2-d0e8-4248-b0e0-bd849f3cc99c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:07.657785 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58d9fa2-d0e8-4248-b0e0-bd849f3cc99c service cinder] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:07.666704 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1183/2385] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:49:07 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:49:17.372715 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=80,keystone:INSERT=5 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:49:17.675879 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=21 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:49:27.194795 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cb25f4-a991-4979-a15e-b1eda9b80895 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:27.194795 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cb25f4-a991-4979-a15e-b1eda9b80895 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:27.195268 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cb25f4-a991-4979-a15e-b1eda9b80895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:27.216052 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-95cb25f4-a991-4979-a15e-b1eda9b80895 None None] MFA Rules not processed for user `02ad6cdc4ffe4917a7bbf6814297f980`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:49:27.217336 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-95cb25f4-a991-4979-a15e-b1eda9b80895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:49:27.284493 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1203/2386] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:49:27 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:49:37.277225 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=25 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:49:40.617122 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58d9fa2-d0e8-4248-b0e0-bd849f3cc99c service cinder] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:40.618628 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-f58d9fa2-d0e8-4248-b0e0-bd849f3cc99c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:49:40.630315 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58d9fa2-d0e8-4248-b0e0-bd849f3cc99c service cinder] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:40.634293 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d16145-5468-46fa-a678-0899b231c669 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36639dce4f97412f89d71367f650b901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:40.635081 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d16145-5468-46fa-a678-0899b231c669 service cinder] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:40.635548 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d16145-5468-46fa-a678-0899b231c669 service cinder] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:40.635978 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d16145-5468-46fa-a678-0899b231c669 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:40.638310 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-37d16145-5468-46fa-a678-0899b231c669 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:49:40.702490 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d16145-5468-46fa-a678-0899b231c669 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:40.703946 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d16145-5468-46fa-a678-0899b231c669 service cinder] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:40.712805 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1184/2387] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:49:40 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:49:50.708779 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=28 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:49:56.111980 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0647ba83-8444-4add-9ed7-3e2195e494c6 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:56.112510 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0647ba83-8444-4add-9ed7-3e2195e494c6 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:56.112510 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0647ba83-8444-4add-9ed7-3e2195e494c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:56.147624 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-0647ba83-8444-4add-9ed7-3e2195e494c6 None None] MFA Rules not processed for user `ea70acd744994250a2905b5dd7163dfb`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:49:56.161841 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-0647ba83-8444-4add-9ed7-3e2195e494c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:49:56.231085 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1204/2388] 10.222.0.47 () {58 vars in 1009 bytes} [Wed Aug 24 06:49:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:49:56.356240 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d16145-5468-46fa-a678-0899b231c669 service cinder] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:49:56.357741 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-37d16145-5468-46fa-a678-0899b231c669 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:49:56.434852 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d16145-5468-46fa-a678-0899b231c669 service cinder] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:49:56.441565 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb1ed6a-b9bd-4ad3-bdfd-43ed7a40ce5d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f1cbaf2cd6427585f298c9d58c319a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:49:56.442186 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb1ed6a-b9bd-4ad3-bdfd-43ed7a40ce5d service swift] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:49:56.442470 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb1ed6a-b9bd-4ad3-bdfd-43ed7a40ce5d service swift] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:49:56.442741 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb1ed6a-b9bd-4ad3-bdfd-43ed7a40ce5d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:49:56.453006 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb1ed6a-b9bd-4ad3-bdfd-43ed7a40ce5d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:49:56.454837 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb1ed6a-b9bd-4ad3-bdfd-43ed7a40ce5d service swift] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:49:56.464719 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1185/2389] 10.222.0.47 () {60 vars in 1384 bytes} [Wed Aug 24 06:49:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 06:50:00.985574 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07a9849-3551-484c-ab4c-ae7433a34907 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:00.986030 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07a9849-3551-484c-ab4c-ae7433a34907 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:00.986030 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07a9849-3551-484c-ab4c-ae7433a34907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:01.011107 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-c07a9849-3551-484c-ab4c-ae7433a34907 None None] MFA Rules not processed for user `594e9324c44540568a81fd5fdb980efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:01.012348 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-c07a9849-3551-484c-ab4c-ae7433a34907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:01.030396 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1205/2390] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:50:00 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:50:06.460781 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=32 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:11.005654 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=35 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:16.508553 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb1ed6a-b9bd-4ad3-bdfd-43ed7a40ce5d service swift] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:16.572146 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb1ed6a-b9bd-4ad3-bdfd-43ed7a40ce5d service swift] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:16.576562 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc4f54d-a4f6-47e9-92b8-037f6226c6af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:16.579647 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc4f54d-a4f6-47e9-92b8-037f6226c6af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:16.579647 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc4f54d-a4f6-47e9-92b8-037f6226c6af admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:16.579647 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc4f54d-a4f6-47e9-92b8-037f6226c6af admin admin] PATH_INFO: `/v3/users/8113e2d3a2d148fb9c391ed9d5130bcb` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:16.579647 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc4f54d-a4f6-47e9-92b8-037f6226c6af admin admin] RBAC: Authorizing `identity:delete_user(user_id=8113e2d3a2d148fb9c391ed9d5130bcb)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:16.581486 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc4f54d-a4f6-47e9-92b8-037f6226c6af admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:16.787886 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-2dc4f54d-a4f6-47e9-92b8-037f6226c6af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8113e2d3a2d148fb9c391ed9d5130bcb'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:16.788737 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-2dc4f54d-a4f6-47e9-92b8-037f6226c6af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8113e2d3a2d148fb9c391ed9d5130bcb'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:16.789449 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-2dc4f54d-a4f6-47e9-92b8-037f6226c6af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8113e2d3a2d148fb9c391ed9d5130bcb'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:16.797198 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-2dc4f54d-a4f6-47e9-92b8-037f6226c6af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8113e2d3a2d148fb9c391ed9d5130bcb'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:16.880627 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-2dc4f54d-a4f6-47e9-92b8-037f6226c6af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8113e2d3a2d148fb9c391ed9d5130bcb'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:16.907674 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1186/2391] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:50:16 2022] DELETE /identity/v3/users/8113e2d3a2d148fb9c391ed9d5130bcb => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:16.913124 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf66e27-ae96-4db3-843e-6cf9dc4ee6d0 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:16.913706 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf66e27-ae96-4db3-843e-6cf9dc4ee6d0 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:16.913706 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf66e27-ae96-4db3-843e-6cf9dc4ee6d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:16.935832 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-dbf66e27-ae96-4db3-843e-6cf9dc4ee6d0 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:16.937414 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-dbf66e27-ae96-4db3-843e-6cf9dc4ee6d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:17.028183 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1206/2392] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:50:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:50:17.036442 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc4f54d-a4f6-47e9-92b8-037f6226c6af admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:17.038191 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-2dc4f54d-a4f6-47e9-92b8-037f6226c6af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:17.110947 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc4f54d-a4f6-47e9-92b8-037f6226c6af admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:17.115407 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b780a034-290d-470c-912e-42c8d56756bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:17.116030 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b780a034-290d-470c-912e-42c8d56756bd service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:17.116314 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b780a034-290d-470c-912e-42c8d56756bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:17.116610 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b780a034-290d-470c-912e-42c8d56756bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:17.125792 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b780a034-290d-470c-912e-42c8d56756bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:17.127538 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b780a034-290d-470c-912e-42c8d56756bd service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:17.137388 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1187/2393] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:50:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:50:17.626521 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf66e27-ae96-4db3-843e-6cf9dc4ee6d0 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:17.628321 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-dbf66e27-ae96-4db3-843e-6cf9dc4ee6d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:17.644035 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf66e27-ae96-4db3-843e-6cf9dc4ee6d0 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:17.648797 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d643569c-ad75-4782-b87f-30bd7a75b043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:17.649482 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d643569c-ad75-4782-b87f-30bd7a75b043 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:17.649767 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d643569c-ad75-4782-b87f-30bd7a75b043 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:17.650044 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d643569c-ad75-4782-b87f-30bd7a75b043 admin admin] PATH_INFO: `/v3/projects/644d103f13eb45d1a280f51c1ac4b6b4` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:17.651704 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d643569c-ad75-4782-b87f-30bd7a75b043 admin admin] RBAC: Authorizing `identity:delete_project(project_id=644d103f13eb45d1a280f51c1ac4b6b4)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:17.653795 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d643569c-ad75-4782-b87f-30bd7a75b043 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:17.796208 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-d643569c-ad75-4782-b87f-30bd7a75b043 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '644d103f13eb45d1a280f51c1ac4b6b4', 'request_id': 'req-d643569c-ad75-4782-b87f-30bd7a75b043'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:17.938677 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-d643569c-ad75-4782-b87f-30bd7a75b043 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '644d103f13eb45d1a280f51c1ac4b6b4', 'request_id': 'req-d643569c-ad75-4782-b87f-30bd7a75b043'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:17.946378 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-d643569c-ad75-4782-b87f-30bd7a75b043 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '644d103f13eb45d1a280f51c1ac4b6b4', 'request_id': 'req-d643569c-ad75-4782-b87f-30bd7a75b043'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:17.957573 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-d643569c-ad75-4782-b87f-30bd7a75b043 admin admin] The token cache is being invalidate because project 644d103f13eb45d1a280f51c1ac4b6b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81723) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:50:17.957858 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-d643569c-ad75-4782-b87f-30bd7a75b043 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:17.958497 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-d643569c-ad75-4782-b87f-30bd7a75b043 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:17.960022 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1207/2394] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:50:17 2022] DELETE /identity/v3/projects/644d103f13eb45d1a280f51c1ac4b6b4 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:17.964196 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b780a034-290d-470c-912e-42c8d56756bd service neutron] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:17.974283 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b780a034-290d-470c-912e-42c8d56756bd service neutron] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:17.992450 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:18.030779 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b8d804-50fd-4bf8-af31-997bea6f1e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:18.031332 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b8d804-50fd-4bf8-af31-997bea6f1e22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:18.031611 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b8d804-50fd-4bf8-af31-997bea6f1e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:18.031883 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b8d804-50fd-4bf8-af31-997bea6f1e22 admin admin] PATH_INFO: `/v3/users/2325d4e211f44f5da9da53ab362bc31b` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:18.033052 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b8d804-50fd-4bf8-af31-997bea6f1e22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2325d4e211f44f5da9da53ab362bc31b)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:18.035044 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b8d804-50fd-4bf8-af31-997bea6f1e22 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:18.234652 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-e3b8d804-50fd-4bf8-af31-997bea6f1e22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2325d4e211f44f5da9da53ab362bc31b'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:18.235172 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-e3b8d804-50fd-4bf8-af31-997bea6f1e22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2325d4e211f44f5da9da53ab362bc31b'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:18.235788 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-e3b8d804-50fd-4bf8-af31-997bea6f1e22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2325d4e211f44f5da9da53ab362bc31b'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:18.243896 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-e3b8d804-50fd-4bf8-af31-997bea6f1e22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2325d4e211f44f5da9da53ab362bc31b'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:18.389936 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-e3b8d804-50fd-4bf8-af31-997bea6f1e22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2325d4e211f44f5da9da53ab362bc31b'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:18.407890 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1188/2395] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:50:17 2022] DELETE /identity/v3/users/2325d4e211f44f5da9da53ab362bc31b => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:18.849696 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d643569c-ad75-4782-b87f-30bd7a75b043 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:18.851563 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-d643569c-ad75-4782-b87f-30bd7a75b043 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:18.866787 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d643569c-ad75-4782-b87f-30bd7a75b043 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:18.926254 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3423d1b1-2e31-4356-8fd0-bc49d94cd8ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:18.926992 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3423d1b1-2e31-4356-8fd0-bc49d94cd8ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:18.927215 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3423d1b1-2e31-4356-8fd0-bc49d94cd8ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:18.927486 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3423d1b1-2e31-4356-8fd0-bc49d94cd8ea admin admin] PATH_INFO: `/v3/projects/3df6c7b57ff24b4a9be7130112eb9b3f` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:18.928616 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3423d1b1-2e31-4356-8fd0-bc49d94cd8ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=3df6c7b57ff24b4a9be7130112eb9b3f)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:18.930608 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3423d1b1-2e31-4356-8fd0-bc49d94cd8ea admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:19.052975 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:19.070016 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-3423d1b1-2e31-4356-8fd0-bc49d94cd8ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3df6c7b57ff24b4a9be7130112eb9b3f', 'request_id': 'req-3423d1b1-2e31-4356-8fd0-bc49d94cd8ea'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:19.144540 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-3423d1b1-2e31-4356-8fd0-bc49d94cd8ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3df6c7b57ff24b4a9be7130112eb9b3f', 'request_id': 'req-3423d1b1-2e31-4356-8fd0-bc49d94cd8ea'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:19.151364 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-3423d1b1-2e31-4356-8fd0-bc49d94cd8ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3df6c7b57ff24b4a9be7130112eb9b3f', 'request_id': 'req-3423d1b1-2e31-4356-8fd0-bc49d94cd8ea'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:19.163580 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-3423d1b1-2e31-4356-8fd0-bc49d94cd8ea admin admin] The token cache is being invalidate because project 3df6c7b57ff24b4a9be7130112eb9b3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81723) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:50:19.163789 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-3423d1b1-2e31-4356-8fd0-bc49d94cd8ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:19.164467 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-3423d1b1-2e31-4356-8fd0-bc49d94cd8ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:19.165998 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1208/2396] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:50:18 2022] DELETE /identity/v3/projects/3df6c7b57ff24b4a9be7130112eb9b3f => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:19.170903 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ad0b3d-d39f-45fb-8f52-64743c69e9f0 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:19.171463 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ad0b3d-d39f-45fb-8f52-64743c69e9f0 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:19.171576 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ad0b3d-d39f-45fb-8f52-64743c69e9f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:19.191389 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-c0ad0b3d-d39f-45fb-8f52-64743c69e9f0 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:19.196655 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-c0ad0b3d-d39f-45fb-8f52-64743c69e9f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:19.253081 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1189/2397] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:50:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:50:19.261434 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de8279-f8f8-45e6-9b48-a42a2b10257b None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:19.261729 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de8279-f8f8-45e6-9b48-a42a2b10257b None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:19.262056 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de8279-f8f8-45e6-9b48-a42a2b10257b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:19.282840 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-a1de8279-f8f8-45e6-9b48-a42a2b10257b None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:19.285027 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-a1de8279-f8f8-45e6-9b48-a42a2b10257b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:19.293068 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1209/2398] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:50:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:50:19.297521 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ad0b3d-d39f-45fb-8f52-64743c69e9f0 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:19.310684 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ad0b3d-d39f-45fb-8f52-64743c69e9f0 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:19.315835 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c555968-ebc0-4a22-af11-15aa799696b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:19.317230 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c555968-ebc0-4a22-af11-15aa799696b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:19.317230 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c555968-ebc0-4a22-af11-15aa799696b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:19.317230 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c555968-ebc0-4a22-af11-15aa799696b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:19.317710 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c555968-ebc0-4a22-af11-15aa799696b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:50:19.318014 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c555968-ebc0-4a22-af11-15aa799696b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:19.319610 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c555968-ebc0-4a22-af11-15aa799696b3 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:19.332487 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1190/2399] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 24 06:50:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:50:19.336693 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1de8279-f8f8-45e6-9b48-a42a2b10257b None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:19.347081 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1de8279-f8f8-45e6-9b48-a42a2b10257b None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:19.352002 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7169ff8-7e58-4743-af79-46fc66f4cdc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:19.352706 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7169ff8-7e58-4743-af79-46fc66f4cdc3 admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:19.353065 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7169ff8-7e58-4743-af79-46fc66f4cdc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:19.353451 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7169ff8-7e58-4743-af79-46fc66f4cdc3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:19.354140 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7169ff8-7e58-4743-af79-46fc66f4cdc3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:19.357394 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7169ff8-7e58-4743-af79-46fc66f4cdc3 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:19.462978 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1210/2400] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:50:19 2022] POST /identity/v3/projects => generated 381 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:50:19.464260 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c555968-ebc0-4a22-af11-15aa799696b3 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:19.476077 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c555968-ebc0-4a22-af11-15aa799696b3 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:19.490074 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461178b3-1e3b-46d7-9c92-712e3314b3e7 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:19.490705 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461178b3-1e3b-46d7-9c92-712e3314b3e7 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:19.491013 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461178b3-1e3b-46d7-9c92-712e3314b3e7 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:19.492402 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1211/2401] 10.222.0.47 () {54 vars in 921 bytes} [Wed Aug 24 06:50:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 06:50:19.499279 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d3c0d5-76ed-4f8e-a580-89eae9b14e5c None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:19.499560 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d3c0d5-76ed-4f8e-a580-89eae9b14e5c None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:19.499837 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d3c0d5-76ed-4f8e-a580-89eae9b14e5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:19.505676 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:19.535546 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-c5d3c0d5-76ed-4f8e-a580-89eae9b14e5c None None] MFA Rules not processed for user `1f4f49c1b08c460f8ba6c451c9c92c2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:19.537139 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-c5d3c0d5-76ed-4f8e-a580-89eae9b14e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:19.541562 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d7b4b6-111f-403b-b44a-06ded30a805e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:19.542197 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d7b4b6-111f-403b-b44a-06ded30a805e admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:19.542571 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d7b4b6-111f-403b-b44a-06ded30a805e admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:19.542776 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d7b4b6-111f-403b-b44a-06ded30a805e admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:19.543570 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d7b4b6-111f-403b-b44a-06ded30a805e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:19.545534 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d7b4b6-111f-403b-b44a-06ded30a805e admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:19.626946 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1212/2402] 10.222.0.47 () {58 vars in 1018 bytes} [Wed Aug 24 06:50:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:50:19.638948 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d3c0d5-76ed-4f8e-a580-89eae9b14e5c None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:19.640362 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-c5d3c0d5-76ed-4f8e-a580-89eae9b14e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:19.653424 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1191/2403] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:50:19 2022] POST /identity/v3/users => generated 291 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:50:19.661530 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d7b4b6-111f-403b-b44a-06ded30a805e admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:19.671869 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d7b4b6-111f-403b-b44a-06ded30a805e admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:19.678505 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96d5959-3a57-4010-bd43-3e4a00f85b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:19.679113 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d5959-3a57-4010-bd43-3e4a00f85b32 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:19.679391 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d5959-3a57-4010-bd43-3e4a00f85b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:19.679664 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d5959-3a57-4010-bd43-3e4a00f85b32 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:19.680353 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96d5959-3a57-4010-bd43-3e4a00f85b32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:19.682116 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96d5959-3a57-4010-bd43-3e4a00f85b32 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:19.692984 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1192/2404] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:50:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:50:19.697293 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96d5959-3a57-4010-bd43-3e4a00f85b32 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:19.703344 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d3c0d5-76ed-4f8e-a580-89eae9b14e5c None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:19.707720 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400087b7-2b5c-4c20-9d27-05a94ae11ec9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b331b6ddfe46aeae7b52fcbbc27771', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:19.708128 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96d5959-3a57-4010-bd43-3e4a00f85b32 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:19.708355 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400087b7-2b5c-4c20-9d27-05a94ae11ec9 service glance] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:19.708650 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400087b7-2b5c-4c20-9d27-05a94ae11ec9 service glance] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:19.708929 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400087b7-2b5c-4c20-9d27-05a94ae11ec9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:19.713441 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785adbfc-c5f2-4468-811f-355fbaf7382b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:19.714034 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785adbfc-c5f2-4468-811f-355fbaf7382b admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:19.714317 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785adbfc-c5f2-4468-811f-355fbaf7382b admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:19.714596 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785adbfc-c5f2-4468-811f-355fbaf7382b admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:19.715238 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785adbfc-c5f2-4468-811f-355fbaf7382b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:19.716816 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785adbfc-c5f2-4468-811f-355fbaf7382b admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:19.719974 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400087b7-2b5c-4c20-9d27-05a94ae11ec9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:19.721503 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400087b7-2b5c-4c20-9d27-05a94ae11ec9 service glance] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:19.727489 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:19.727855 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1193/2405] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:50:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:50:19.732073 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785adbfc-c5f2-4468-811f-355fbaf7382b admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:19.733782 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1213/2406] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:50:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:50:19.743860 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785adbfc-c5f2-4468-811f-355fbaf7382b admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:19.748759 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a027aa16-07e3-4f8a-815f-7b598dde47d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:19.749636 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a027aa16-07e3-4f8a-815f-7b598dde47d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:19.749845 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a027aa16-07e3-4f8a-815f-7b598dde47d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:19.750111 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a027aa16-07e3-4f8a-815f-7b598dde47d1 admin admin] PATH_INFO: `/v3/projects/794bcaecca5047c6b21a2ed733f1a579/users/db61fd827f8b4b55a0631f8f8154f5df/roles/e1f9b167c51c4a388cbc9594db8a465d` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:19.767603 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a027aa16-07e3-4f8a-815f-7b598dde47d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=794bcaecca5047c6b21a2ed733f1a579, user_id=db61fd827f8b4b55a0631f8f8154f5df, role_id=e1f9b167c51c4a388cbc9594db8a465d)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:19.770037 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a027aa16-07e3-4f8a-815f-7b598dde47d1 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:19.865356 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1194/2407] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:50:19 2022] PUT /identity/v3/projects/794bcaecca5047c6b21a2ed733f1a579/users/db61fd827f8b4b55a0631f8f8154f5df/roles/e1f9b167c51c4a388cbc9594db8a465d => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:50:19.875896 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336e1c18-56f7-47e8-8d56-b66b5a255246 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:19.876306 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336e1c18-56f7-47e8-8d56-b66b5a255246 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:19.877361 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336e1c18-56f7-47e8-8d56-b66b5a255246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:19.905410 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-336e1c18-56f7-47e8-8d56-b66b5a255246 None None] MFA Rules not processed for user `db61fd827f8b4b55a0631f8f8154f5df`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:19.906634 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-336e1c18-56f7-47e8-8d56-b66b5a255246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:19.973097 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1214/2408] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:50:19 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:50:19.978326 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a027aa16-07e3-4f8a-815f-7b598dde47d1 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:19.997964 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a1bda-d54c-4463-971a-04d3bc081595 None None] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:19.998372 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a1bda-d54c-4463-971a-04d3bc081595 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:19.998747 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a1bda-d54c-4463-971a-04d3bc081595 None None] PATH_INFO: `/` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:20.000437 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1215/2409] 10.222.0.47 () {54 vars in 921 bytes} [Wed Aug 24 06:50:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 06:50:20.005810 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6081fd0e-b737-41fb-9112-ecaa63a7a218 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:20.006186 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6081fd0e-b737-41fb-9112-ecaa63a7a218 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:20.006552 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6081fd0e-b737-41fb-9112-ecaa63a7a218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:20.027666 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-6081fd0e-b737-41fb-9112-ecaa63a7a218 None None] MFA Rules not processed for user `f0ba3f03ec15475ab6ebf64994ad742c`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:20.030065 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-6081fd0e-b737-41fb-9112-ecaa63a7a218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:20.060956 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a027aa16-07e3-4f8a-815f-7b598dde47d1 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:20.068067 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834ee420-7a84-4e92-b8cf-c618be4f02b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:20.068807 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834ee420-7a84-4e92-b8cf-c618be4f02b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:20.069092 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834ee420-7a84-4e92-b8cf-c618be4f02b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:20.069422 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834ee420-7a84-4e92-b8cf-c618be4f02b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:20.070074 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834ee420-7a84-4e92-b8cf-c618be4f02b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:20.072027 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834ee420-7a84-4e92-b8cf-c618be4f02b1 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:20.097022 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1216/2410] 10.222.0.47 () {58 vars in 1018 bytes} [Wed Aug 24 06:50:20 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:50:20.109049 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6081fd0e-b737-41fb-9112-ecaa63a7a218 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:20.110498 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-6081fd0e-b737-41fb-9112-ecaa63a7a218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:20.121577 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6081fd0e-b737-41fb-9112-ecaa63a7a218 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:20.125297 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2287199-9f78-4ff3-be45-7c0904d7e2b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:20.125897 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2287199-9f78-4ff3-be45-7c0904d7e2b4 service neutron] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:20.126226 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2287199-9f78-4ff3-be45-7c0904d7e2b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:20.126634 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2287199-9f78-4ff3-be45-7c0904d7e2b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:20.134004 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2287199-9f78-4ff3-be45-7c0904d7e2b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:20.135561 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2287199-9f78-4ff3-be45-7c0904d7e2b4 service neutron] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:20.144641 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1217/2411] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:50:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:50:20.197714 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1195/2412] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:50:19 2022] POST /identity/v3/projects => generated 379 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:50:20.201882 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2287199-9f78-4ff3-be45-7c0904d7e2b4 service neutron] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:20.215061 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2287199-9f78-4ff3-be45-7c0904d7e2b4 service neutron] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:20.231829 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27784198-ea93-452c-a36f-e01d8d59a4b4 None None] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:20.232112 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27784198-ea93-452c-a36f-e01d8d59a4b4 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:20.232388 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27784198-ea93-452c-a36f-e01d8d59a4b4 None None] PATH_INFO: `/` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:20.233967 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1196/2413] 10.222.0.47 () {54 vars in 921 bytes} [Wed Aug 24 06:50:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 06:50:20.240079 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ec6bb2-f3ba-42a1-b2cf-6a96f56a9c49 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:20.240446 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ec6bb2-f3ba-42a1-b2cf-6a96f56a9c49 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:20.240786 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ec6bb2-f3ba-42a1-b2cf-6a96f56a9c49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:20.260699 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-c9ec6bb2-f3ba-42a1-b2cf-6a96f56a9c49 None None] MFA Rules not processed for user `1f4f49c1b08c460f8ba6c451c9c92c2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:20.261949 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-c9ec6bb2-f3ba-42a1-b2cf-6a96f56a9c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:20.273914 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de98ee6d-ecf7-4b13-99be-a3303a939095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:20.274552 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de98ee6d-ecf7-4b13-99be-a3303a939095 admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:20.274927 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de98ee6d-ecf7-4b13-99be-a3303a939095 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:20.275213 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de98ee6d-ecf7-4b13-99be-a3303a939095 admin admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:20.275843 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de98ee6d-ecf7-4b13-99be-a3303a939095 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:20.278354 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de98ee6d-ecf7-4b13-99be-a3303a939095 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:20.316952 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:20.334622 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1197/2414] 10.222.0.47 () {58 vars in 1018 bytes} [Wed Aug 24 06:50:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:50:20.345616 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ec6bb2-f3ba-42a1-b2cf-6a96f56a9c49 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:20.347435 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-c9ec6bb2-f3ba-42a1-b2cf-6a96f56a9c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:20.378329 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1218/2415] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:50:20 2022] POST /identity/v3/users => generated 296 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:50:20.378329 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de98ee6d-ecf7-4b13-99be-a3303a939095 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:20.387438 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de98ee6d-ecf7-4b13-99be-a3303a939095 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:20.392151 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3472de-15da-4995-89ad-6438a6745ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:20.393551 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3472de-15da-4995-89ad-6438a6745ea5 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:20.393797 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3472de-15da-4995-89ad-6438a6745ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:20.394107 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3472de-15da-4995-89ad-6438a6745ea5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:20.394738 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3472de-15da-4995-89ad-6438a6745ea5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:20.396336 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3472de-15da-4995-89ad-6438a6745ea5 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:20.406743 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1219/2416] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:50:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:50:20.407354 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ec6bb2-f3ba-42a1-b2cf-6a96f56a9c49 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:20.412763 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3472de-15da-4995-89ad-6438a6745ea5 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:20.417493 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f224f8a-ed20-4dc7-a76e-62ba3374ec81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:20.418129 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f224f8a-ed20-4dc7-a76e-62ba3374ec81 service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:20.418411 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f224f8a-ed20-4dc7-a76e-62ba3374ec81 service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:20.418687 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f224f8a-ed20-4dc7-a76e-62ba3374ec81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:20.422824 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:20.428482 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3472de-15da-4995-89ad-6438a6745ea5 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:20.434158 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f224f8a-ed20-4dc7-a76e-62ba3374ec81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:20.438434 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe557aa-dd20-48ab-bf39-5e50eb4a427e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:20.438434 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe557aa-dd20-48ab-bf39-5e50eb4a427e admin admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:20.438434 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe557aa-dd20-48ab-bf39-5e50eb4a427e admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:20.438434 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe557aa-dd20-48ab-bf39-5e50eb4a427e admin admin] PATH_INFO: `/v3/projects/70bb05b7c70b4f2cb88d884b5790743f/users/65f581b3483c41ae9ce552f3aab1a450/roles/486144ad1bcd417e86acdbc1d14327bf` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:20.440512 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f224f8a-ed20-4dc7-a76e-62ba3374ec81 service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:20.451002 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1198/2417] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:50:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:50:20.452504 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe557aa-dd20-48ab-bf39-5e50eb4a427e admin admin] RBAC: Authorizing `identity:create_grant(project_id=70bb05b7c70b4f2cb88d884b5790743f, user_id=65f581b3483c41ae9ce552f3aab1a450, role_id=486144ad1bcd417e86acdbc1d14327bf)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:20.456033 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe557aa-dd20-48ab-bf39-5e50eb4a427e admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:20.528995 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1220/2418] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:50:20 2022] PUT /identity/v3/projects/70bb05b7c70b4f2cb88d884b5790743f/users/65f581b3483c41ae9ce552f3aab1a450/roles/486144ad1bcd417e86acdbc1d14327bf => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:50:20.533083 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f224f8a-ed20-4dc7-a76e-62ba3374ec81 service neutron] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:20.544369 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f224f8a-ed20-4dc7-a76e-62ba3374ec81 service neutron] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:20.633571 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6632eca5-0481-4eb9-83eb-4e7aa7d6196d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:20.634330 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6632eca5-0481-4eb9-83eb-4e7aa7d6196d admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:20.634644 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6632eca5-0481-4eb9-83eb-4e7aa7d6196d admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:20.634914 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6632eca5-0481-4eb9-83eb-4e7aa7d6196d admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:20.635555 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6632eca5-0481-4eb9-83eb-4e7aa7d6196d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:20.637162 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6632eca5-0481-4eb9-83eb-4e7aa7d6196d admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:20.650898 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1199/2419] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:50:20 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:50:20.662092 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe557aa-dd20-48ab-bf39-5e50eb4a427e admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:20.679321 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe557aa-dd20-48ab-bf39-5e50eb4a427e admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:20.687033 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fdf900-4fdc-4911-a247-6ca995822437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:20.687932 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fdf900-4fdc-4911-a247-6ca995822437 admin admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:20.688218 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fdf900-4fdc-4911-a247-6ca995822437 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:20.688495 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fdf900-4fdc-4911-a247-6ca995822437 admin admin] PATH_INFO: `/v3/projects/70bb05b7c70b4f2cb88d884b5790743f/users/65f581b3483c41ae9ce552f3aab1a450/roles/486144ad1bcd417e86acdbc1d14327bf` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:20.691337 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fdf900-4fdc-4911-a247-6ca995822437 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70bb05b7c70b4f2cb88d884b5790743f, user_id=65f581b3483c41ae9ce552f3aab1a450, role_id=486144ad1bcd417e86acdbc1d14327bf)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:20.694235 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fdf900-4fdc-4911-a247-6ca995822437 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:20.712033 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1221/2420] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:50:20 2022] PUT /identity/v3/projects/70bb05b7c70b4f2cb88d884b5790743f/users/65f581b3483c41ae9ce552f3aab1a450/roles/486144ad1bcd417e86acdbc1d14327bf => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:20.723668 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5ea5f0-dfcf-4bc8-8712-fdb1a8589a13 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:20.724367 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5ea5f0-dfcf-4bc8-8712-fdb1a8589a13 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:20.724367 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5ea5f0-dfcf-4bc8-8712-fdb1a8589a13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:20.756698 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-5b5ea5f0-dfcf-4bc8-8712-fdb1a8589a13 None None] MFA Rules not processed for user `65f581b3483c41ae9ce552f3aab1a450`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:20.760084 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-5b5ea5f0-dfcf-4bc8-8712-fdb1a8589a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:20.834480 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1200/2421] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:50:20 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:50:20.847144 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fdf900-4fdc-4911-a247-6ca995822437 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:20.848888 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-80fdf900-4fdc-4911-a247-6ca995822437 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:20.910039 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fdf900-4fdc-4911-a247-6ca995822437 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:20.913917 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a238aa-a157-4280-8f4c-38b2a532ccfb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36639dce4f97412f89d71367f650b901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:20.914527 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a238aa-a157-4280-8f4c-38b2a532ccfb service cinder] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:20.914829 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a238aa-a157-4280-8f4c-38b2a532ccfb service cinder] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:20.915112 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a238aa-a157-4280-8f4c-38b2a532ccfb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:20.925279 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a238aa-a157-4280-8f4c-38b2a532ccfb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:20.926735 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a238aa-a157-4280-8f4c-38b2a532ccfb service cinder] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:20.936387 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1222/2422] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:50:20 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:50:21.176007 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5ea5f0-dfcf-4bc8-8712-fdb1a8589a13 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:21.185964 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5ea5f0-dfcf-4bc8-8712-fdb1a8589a13 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:21.190444 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2f4c24-c343-49ed-8228-da7b3136ff35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36639dce4f97412f89d71367f650b901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:21.191055 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2f4c24-c343-49ed-8228-da7b3136ff35 service cinder] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:21.191360 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2f4c24-c343-49ed-8228-da7b3136ff35 service cinder] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:21.191633 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2f4c24-c343-49ed-8228-da7b3136ff35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:21.201781 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:21.253139 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2f4c24-c343-49ed-8228-da7b3136ff35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:21.254636 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2f4c24-c343-49ed-8228-da7b3136ff35 service cinder] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:21.263972 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1201/2423] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:50:21 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:50:25.288334 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f0b1f-a394-4692-8d40-6f0bb50112f7 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:25.288334 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f0b1f-a394-4692-8d40-6f0bb50112f7 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:25.288334 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f0b1f-a394-4692-8d40-6f0bb50112f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:25.307520 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-515f0b1f-a394-4692-8d40-6f0bb50112f7 None None] MFA Rules not processed for user `ea70acd744994250a2905b5dd7163dfb`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:25.309226 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-515f0b1f-a394-4692-8d40-6f0bb50112f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:25.385857 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1223/2424] 10.222.0.47 () {58 vars in 1009 bytes} [Wed Aug 24 06:50:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:50:25.398500 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2f4c24-c343-49ed-8228-da7b3136ff35 service cinder] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:25.402666 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-8c2f4c24-c343-49ed-8228-da7b3136ff35 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:25.493882 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2f4c24-c343-49ed-8228-da7b3136ff35 service cinder] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:25.500221 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170024df-af6c-4d9a-8bd2-0882dece1d5e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f1cbaf2cd6427585f298c9d58c319a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:25.501039 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170024df-af6c-4d9a-8bd2-0882dece1d5e service swift] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:25.501443 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170024df-af6c-4d9a-8bd2-0882dece1d5e service swift] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:25.501926 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170024df-af6c-4d9a-8bd2-0882dece1d5e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:25.513356 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170024df-af6c-4d9a-8bd2-0882dece1d5e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:25.515097 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170024df-af6c-4d9a-8bd2-0882dece1d5e service swift] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:25.530780 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1202/2425] 10.222.0.47 () {60 vars in 1384 bytes} [Wed Aug 24 06:50:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 06:50:31.515316 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515f0b1f-a394-4692-8d40-6f0bb50112f7 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:31.569773 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515f0b1f-a394-4692-8d40-6f0bb50112f7 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:31.573765 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0c67df-1f8a-416d-bd2a-94e15042b307 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:31.574398 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0c67df-1f8a-416d-bd2a-94e15042b307 service neutron] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:31.574675 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0c67df-1f8a-416d-bd2a-94e15042b307 service neutron] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:31.574945 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0c67df-1f8a-416d-bd2a-94e15042b307 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:31.576793 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-0d0c67df-1f8a-416d-bd2a-94e15042b307 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:31.609135 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:31.652719 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0c67df-1f8a-416d-bd2a-94e15042b307 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:31.654211 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0c67df-1f8a-416d-bd2a-94e15042b307 service neutron] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:31.665312 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1224/2426] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:50:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:50:32.458641 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170024df-af6c-4d9a-8bd2-0882dece1d5e service swift] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:32.467945 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170024df-af6c-4d9a-8bd2-0882dece1d5e service swift] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:32.471977 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fe77a8-70ab-4044-83b0-e4df69f4ed78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:32.472783 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fe77a8-70ab-4044-83b0-e4df69f4ed78 service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:32.473065 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fe77a8-70ab-4044-83b0-e4df69f4ed78 service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:32.473376 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fe77a8-70ab-4044-83b0-e4df69f4ed78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:32.475263 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-f0fe77a8-70ab-4044-83b0-e4df69f4ed78 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:32.489232 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fe77a8-70ab-4044-83b0-e4df69f4ed78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:32.490645 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fe77a8-70ab-4044-83b0-e4df69f4ed78 service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:32.501833 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1203/2427] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:50:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:50:41.681442 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=20 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:42.496757 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=60 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:43.664518 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0c67df-1f8a-416d-bd2a-94e15042b307 service neutron] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:43.740240 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0c67df-1f8a-416d-bd2a-94e15042b307 service neutron] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:43.744898 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcd2a2a-4638-4168-85f4-168865d21aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:43.745579 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcd2a2a-4638-4168-85f4-168865d21aad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:43.745850 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcd2a2a-4638-4168-85f4-168865d21aad admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:43.746119 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcd2a2a-4638-4168-85f4-168865d21aad admin admin] PATH_INFO: `/v3/users/db61fd827f8b4b55a0631f8f8154f5df` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:43.747243 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcd2a2a-4638-4168-85f4-168865d21aad admin admin] RBAC: Authorizing `identity:delete_user(user_id=db61fd827f8b4b55a0631f8f8154f5df)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:43.749240 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcd2a2a-4638-4168-85f4-168865d21aad admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:43.953006 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-5dcd2a2a-4638-4168-85f4-168865d21aad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db61fd827f8b4b55a0631f8f8154f5df'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:43.953986 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-5dcd2a2a-4638-4168-85f4-168865d21aad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db61fd827f8b4b55a0631f8f8154f5df'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:43.954588 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-5dcd2a2a-4638-4168-85f4-168865d21aad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db61fd827f8b4b55a0631f8f8154f5df'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:44.052850 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-5dcd2a2a-4638-4168-85f4-168865d21aad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db61fd827f8b4b55a0631f8f8154f5df'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:44.067838 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-5dcd2a2a-4638-4168-85f4-168865d21aad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db61fd827f8b4b55a0631f8f8154f5df'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:44.082416 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1225/2428] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:50:43 2022] DELETE /identity/v3/users/db61fd827f8b4b55a0631f8f8154f5df => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:44.087821 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b80120-3950-411d-b334-f52b8132e06d None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:44.090800 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b80120-3950-411d-b334-f52b8132e06d None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:44.090800 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b80120-3950-411d-b334-f52b8132e06d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:44.111014 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-32b80120-3950-411d-b334-f52b8132e06d None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:44.111774 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-32b80120-3950-411d-b334-f52b8132e06d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:44.173253 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1204/2429] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:50:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:50:44.184425 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcd2a2a-4638-4168-85f4-168865d21aad admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:44.186024 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-5dcd2a2a-4638-4168-85f4-168865d21aad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:44.249223 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcd2a2a-4638-4168-85f4-168865d21aad admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:44.252855 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839b7670-1cfd-4e2d-b7c9-530a0689d316 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:44.254242 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839b7670-1cfd-4e2d-b7c9-530a0689d316 service neutron] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:44.254522 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839b7670-1cfd-4e2d-b7c9-530a0689d316 service neutron] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:44.254804 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839b7670-1cfd-4e2d-b7c9-530a0689d316 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:44.264382 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839b7670-1cfd-4e2d-b7c9-530a0689d316 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:44.265875 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839b7670-1cfd-4e2d-b7c9-530a0689d316 service neutron] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:44.276242 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1226/2430] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:50:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:50:44.734416 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b80120-3950-411d-b334-f52b8132e06d None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:44.735872 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-32b80120-3950-411d-b334-f52b8132e06d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:44.748797 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b80120-3950-411d-b334-f52b8132e06d None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:44.754309 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f39a18e-e072-464d-b0d3-ca4c506bd29f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:44.754729 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f39a18e-e072-464d-b0d3-ca4c506bd29f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:44.755002 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f39a18e-e072-464d-b0d3-ca4c506bd29f admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:44.755457 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f39a18e-e072-464d-b0d3-ca4c506bd29f admin admin] PATH_INFO: `/v3/projects/794bcaecca5047c6b21a2ed733f1a579` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:44.756810 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f39a18e-e072-464d-b0d3-ca4c506bd29f admin admin] RBAC: Authorizing `identity:delete_project(project_id=794bcaecca5047c6b21a2ed733f1a579)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:44.758759 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f39a18e-e072-464d-b0d3-ca4c506bd29f admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:44.892788 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-7f39a18e-e072-464d-b0d3-ca4c506bd29f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '794bcaecca5047c6b21a2ed733f1a579', 'request_id': 'req-7f39a18e-e072-464d-b0d3-ca4c506bd29f'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:44.900843 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-7f39a18e-e072-464d-b0d3-ca4c506bd29f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '794bcaecca5047c6b21a2ed733f1a579', 'request_id': 'req-7f39a18e-e072-464d-b0d3-ca4c506bd29f'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:45.046838 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-7f39a18e-e072-464d-b0d3-ca4c506bd29f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '794bcaecca5047c6b21a2ed733f1a579', 'request_id': 'req-7f39a18e-e072-464d-b0d3-ca4c506bd29f'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:45.059941 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-7f39a18e-e072-464d-b0d3-ca4c506bd29f admin admin] The token cache is being invalidate because project 794bcaecca5047c6b21a2ed733f1a579 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:50:45.060229 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-7f39a18e-e072-464d-b0d3-ca4c506bd29f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:45.060901 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-7f39a18e-e072-464d-b0d3-ca4c506bd29f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:45.062462 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1205/2431] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:50:44 2022] DELETE /identity/v3/projects/794bcaecca5047c6b21a2ed733f1a579 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:45.067362 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839b7670-1cfd-4e2d-b7c9-530a0689d316 service neutron] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:45.077078 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839b7670-1cfd-4e2d-b7c9-530a0689d316 service neutron] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:45.097655 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:45.142624 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c33ee39-c227-471c-bd1d-0361d721eb9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:45.143372 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c33ee39-c227-471c-bd1d-0361d721eb9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:45.144368 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c33ee39-c227-471c-bd1d-0361d721eb9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:45.145140 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c33ee39-c227-471c-bd1d-0361d721eb9b admin admin] PATH_INFO: `/v3/users/65f581b3483c41ae9ce552f3aab1a450` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:45.146204 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c33ee39-c227-471c-bd1d-0361d721eb9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=65f581b3483c41ae9ce552f3aab1a450)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:45.148202 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c33ee39-c227-471c-bd1d-0361d721eb9b admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:45.388071 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-7c33ee39-c227-471c-bd1d-0361d721eb9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65f581b3483c41ae9ce552f3aab1a450'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:45.388733 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-7c33ee39-c227-471c-bd1d-0361d721eb9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65f581b3483c41ae9ce552f3aab1a450'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:45.389368 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-7c33ee39-c227-471c-bd1d-0361d721eb9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65f581b3483c41ae9ce552f3aab1a450'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:45.469932 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-7c33ee39-c227-471c-bd1d-0361d721eb9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65f581b3483c41ae9ce552f3aab1a450'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:45.483267 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-7c33ee39-c227-471c-bd1d-0361d721eb9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65f581b3483c41ae9ce552f3aab1a450'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:45.498139 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1227/2432] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:50:45 2022] DELETE /identity/v3/users/65f581b3483c41ae9ce552f3aab1a450 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:45.946205 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f39a18e-e072-464d-b0d3-ca4c506bd29f admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:45.948488 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-7f39a18e-e072-464d-b0d3-ca4c506bd29f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:45.966580 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f39a18e-e072-464d-b0d3-ca4c506bd29f admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:46.028788 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822b90db-d7b0-4614-b30d-52bc2e0fc8a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:46.029698 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b90db-d7b0-4614-b30d-52bc2e0fc8a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:46.029987 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b90db-d7b0-4614-b30d-52bc2e0fc8a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:46.030283 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b90db-d7b0-4614-b30d-52bc2e0fc8a9 admin admin] PATH_INFO: `/v3/projects/70bb05b7c70b4f2cb88d884b5790743f` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:46.031463 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822b90db-d7b0-4614-b30d-52bc2e0fc8a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70bb05b7c70b4f2cb88d884b5790743f)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:46.033446 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822b90db-d7b0-4614-b30d-52bc2e0fc8a9 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:46.134436 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:46.156636 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-822b90db-d7b0-4614-b30d-52bc2e0fc8a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70bb05b7c70b4f2cb88d884b5790743f', 'request_id': 'req-822b90db-d7b0-4614-b30d-52bc2e0fc8a9'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:46.163784 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-822b90db-d7b0-4614-b30d-52bc2e0fc8a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70bb05b7c70b4f2cb88d884b5790743f', 'request_id': 'req-822b90db-d7b0-4614-b30d-52bc2e0fc8a9'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:46.298636 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-822b90db-d7b0-4614-b30d-52bc2e0fc8a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70bb05b7c70b4f2cb88d884b5790743f', 'request_id': 'req-822b90db-d7b0-4614-b30d-52bc2e0fc8a9'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:46.313075 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-822b90db-d7b0-4614-b30d-52bc2e0fc8a9 admin admin] The token cache is being invalidate because project 70bb05b7c70b4f2cb88d884b5790743f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:50:46.313390 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-822b90db-d7b0-4614-b30d-52bc2e0fc8a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:46.314083 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-822b90db-d7b0-4614-b30d-52bc2e0fc8a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:46.316184 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1206/2433] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:50:45 2022] DELETE /identity/v3/projects/70bb05b7c70b4f2cb88d884b5790743f => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:46.321414 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcb0e99-97a7-4d4e-9b6c-17341f26bbb4 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:46.321938 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcb0e99-97a7-4d4e-9b6c-17341f26bbb4 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:46.321938 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcb0e99-97a7-4d4e-9b6c-17341f26bbb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:46.343322 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-4bcb0e99-97a7-4d4e-9b6c-17341f26bbb4 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:46.350900 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-4bcb0e99-97a7-4d4e-9b6c-17341f26bbb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:46.412922 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1228/2434] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:50:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:50:46.421283 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf210b9-3afb-4321-974e-03b61ec3724e None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:46.421568 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf210b9-3afb-4321-974e-03b61ec3724e None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:46.421841 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf210b9-3afb-4321-974e-03b61ec3724e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:46.443902 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-1bf210b9-3afb-4321-974e-03b61ec3724e None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:46.445078 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-1bf210b9-3afb-4321-974e-03b61ec3724e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:46.453190 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1207/2435] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:50:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:50:46.459022 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcb0e99-97a7-4d4e-9b6c-17341f26bbb4 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:46.470440 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcb0e99-97a7-4d4e-9b6c-17341f26bbb4 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:46.475526 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dbd021-6f05-4464-91ce-3f5b915a6cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:46.476149 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dbd021-6f05-4464-91ce-3f5b915a6cd5 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:46.476436 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dbd021-6f05-4464-91ce-3f5b915a6cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:46.476715 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dbd021-6f05-4464-91ce-3f5b915a6cd5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:46.477503 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dbd021-6f05-4464-91ce-3f5b915a6cd5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:50:46.477806 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dbd021-6f05-4464-91ce-3f5b915a6cd5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:46.479514 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dbd021-6f05-4464-91ce-3f5b915a6cd5 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:46.494280 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1229/2436] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 24 06:50:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:50:46.498969 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf210b9-3afb-4321-974e-03b61ec3724e None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:46.509760 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf210b9-3afb-4321-974e-03b61ec3724e None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:46.514735 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0972b5fe-6e15-4c66-b662-068783ccc02f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:46.515350 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0972b5fe-6e15-4c66-b662-068783ccc02f admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:46.515630 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0972b5fe-6e15-4c66-b662-068783ccc02f admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:46.515912 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0972b5fe-6e15-4c66-b662-068783ccc02f admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:46.516551 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0972b5fe-6e15-4c66-b662-068783ccc02f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:46.519123 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0972b5fe-6e15-4c66-b662-068783ccc02f admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:46.626289 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1208/2437] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:50:46 2022] POST /identity/v3/projects => generated 389 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:50:46.637583 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dbd021-6f05-4464-91ce-3f5b915a6cd5 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:46.649751 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dbd021-6f05-4464-91ce-3f5b915a6cd5 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:46.678764 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:46.714714 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77906af-bf58-44e8-8c82-76d524f44e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:46.715239 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77906af-bf58-44e8-8c82-76d524f44e30 admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:46.715515 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77906af-bf58-44e8-8c82-76d524f44e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:46.715781 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77906af-bf58-44e8-8c82-76d524f44e30 admin admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:46.716404 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77906af-bf58-44e8-8c82-76d524f44e30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:46.718603 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77906af-bf58-44e8-8c82-76d524f44e30 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:46.841808 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1230/2438] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:50:46 2022] POST /identity/v3/users => generated 295 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:50:46.848684 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0972b5fe-6e15-4c66-b662-068783ccc02f admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:46.859753 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0972b5fe-6e15-4c66-b662-068783ccc02f admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:46.865143 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409531e8-b4f7-43b5-9f4d-1a68d3689848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:46.865793 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409531e8-b4f7-43b5-9f4d-1a68d3689848 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:46.866072 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409531e8-b4f7-43b5-9f4d-1a68d3689848 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:46.866345 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409531e8-b4f7-43b5-9f4d-1a68d3689848 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:46.866997 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409531e8-b4f7-43b5-9f4d-1a68d3689848 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:46.868705 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409531e8-b4f7-43b5-9f4d-1a68d3689848 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:46.882563 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1209/2439] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:50:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:50:46.886906 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77906af-bf58-44e8-8c82-76d524f44e30 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:46.898963 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77906af-bf58-44e8-8c82-76d524f44e30 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:46.904627 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d4e834-2e00-4085-b71e-9f713f9e64d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:46.905278 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d4e834-2e00-4085-b71e-9f713f9e64d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:46.905562 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d4e834-2e00-4085-b71e-9f713f9e64d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:46.905837 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d4e834-2e00-4085-b71e-9f713f9e64d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:46.906490 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d4e834-2e00-4085-b71e-9f713f9e64d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:46.908333 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d4e834-2e00-4085-b71e-9f713f9e64d2 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:46.920235 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1231/2440] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:50:46 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:50:46.924966 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409531e8-b4f7-43b5-9f4d-1a68d3689848 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:46.935711 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409531e8-b4f7-43b5-9f4d-1a68d3689848 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:46.940785 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296f2f08-bb9c-41c8-908f-67e6b17c807d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:46.941488 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296f2f08-bb9c-41c8-908f-67e6b17c807d admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:46.942350 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296f2f08-bb9c-41c8-908f-67e6b17c807d admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:46.942350 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296f2f08-bb9c-41c8-908f-67e6b17c807d admin admin] PATH_INFO: `/v3/projects/90fd5a3905e34554a71eedec8ef8d362/users/85168dcd649f46b8a176400531e28057/roles/e1f9b167c51c4a388cbc9594db8a465d` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:46.959985 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296f2f08-bb9c-41c8-908f-67e6b17c807d admin admin] RBAC: Authorizing `identity:create_grant(project_id=90fd5a3905e34554a71eedec8ef8d362, user_id=85168dcd649f46b8a176400531e28057, role_id=e1f9b167c51c4a388cbc9594db8a465d)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:46.962727 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296f2f08-bb9c-41c8-908f-67e6b17c807d admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:47.047572 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1210/2441] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:50:46 2022] PUT /identity/v3/projects/90fd5a3905e34554a71eedec8ef8d362/users/85168dcd649f46b8a176400531e28057/roles/e1f9b167c51c4a388cbc9594db8a465d => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:47.116445 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250984f7-15d0-4994-8758-34b4d73ef12b None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:47.116729 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250984f7-15d0-4994-8758-34b4d73ef12b None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:47.117007 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250984f7-15d0-4994-8758-34b4d73ef12b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:47.149423 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-250984f7-15d0-4994-8758-34b4d73ef12b None None] MFA Rules not processed for user `85168dcd649f46b8a176400531e28057`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:47.153154 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-250984f7-15d0-4994-8758-34b4d73ef12b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:47.259484 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1232/2442] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:50:47 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:50:47.264342 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296f2f08-bb9c-41c8-908f-67e6b17c807d admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:47.342538 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296f2f08-bb9c-41c8-908f-67e6b17c807d admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:47.348023 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c464e6-2893-481a-bca0-e343dcf1159c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:47.349139 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c464e6-2893-481a-bca0-e343dcf1159c admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:47.349698 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c464e6-2893-481a-bca0-e343dcf1159c admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:47.350203 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c464e6-2893-481a-bca0-e343dcf1159c admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:47.351054 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c464e6-2893-481a-bca0-e343dcf1159c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:47.353782 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c464e6-2893-481a-bca0-e343dcf1159c admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:47.461759 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1211/2443] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:50:47 2022] POST /identity/v3/projects => generated 387 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:50:47.466097 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250984f7-15d0-4994-8758-34b4d73ef12b None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:47.477963 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250984f7-15d0-4994-8758-34b4d73ef12b None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:47.552641 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bd24f2-68ae-490d-a4d0-7e87bd9c9f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:47.553371 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bd24f2-68ae-490d-a4d0-7e87bd9c9f56 admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:47.553599 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bd24f2-68ae-490d-a4d0-7e87bd9c9f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:47.553870 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bd24f2-68ae-490d-a4d0-7e87bd9c9f56 admin admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:47.554518 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bd24f2-68ae-490d-a4d0-7e87bd9c9f56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:47.556550 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bd24f2-68ae-490d-a4d0-7e87bd9c9f56 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:47.687656 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1233/2444] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:50:47 2022] POST /identity/v3/users => generated 300 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:50:47.691531 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c464e6-2893-481a-bca0-e343dcf1159c admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:47.708163 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c464e6-2893-481a-bca0-e343dcf1159c admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:47.713122 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf403179-6682-42a1-bf08-44b4260c4b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:47.713775 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf403179-6682-42a1-bf08-44b4260c4b89 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:47.714053 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf403179-6682-42a1-bf08-44b4260c4b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:47.714338 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf403179-6682-42a1-bf08-44b4260c4b89 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:47.714980 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf403179-6682-42a1-bf08-44b4260c4b89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:47.716622 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf403179-6682-42a1-bf08-44b4260c4b89 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:47.729904 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1212/2445] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:50:47 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:50:47.734263 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bd24f2-68ae-490d-a4d0-7e87bd9c9f56 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:47.747920 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bd24f2-68ae-490d-a4d0-7e87bd9c9f56 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:47.752842 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cf4e9c-f078-45cb-8a12-1f17334d61fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:47.753604 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cf4e9c-f078-45cb-8a12-1f17334d61fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:47.753887 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cf4e9c-f078-45cb-8a12-1f17334d61fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:47.754796 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cf4e9c-f078-45cb-8a12-1f17334d61fc admin admin] PATH_INFO: `/v3/projects/b68d5e10933147df870b269dfc7a3932/users/3d8fa87088714a218dd04f86ee890175/roles/486144ad1bcd417e86acdbc1d14327bf` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:47.771935 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cf4e9c-f078-45cb-8a12-1f17334d61fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=b68d5e10933147df870b269dfc7a3932, user_id=3d8fa87088714a218dd04f86ee890175, role_id=486144ad1bcd417e86acdbc1d14327bf)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:47.774886 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cf4e9c-f078-45cb-8a12-1f17334d61fc admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:47.911258 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1234/2446] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:50:47 2022] PUT /identity/v3/projects/b68d5e10933147df870b269dfc7a3932/users/3d8fa87088714a218dd04f86ee890175/roles/486144ad1bcd417e86acdbc1d14327bf => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:47.915993 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf403179-6682-42a1-bf08-44b4260c4b89 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:47.927309 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf403179-6682-42a1-bf08-44b4260c4b89 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:47.940679 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:48.005366 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5e9fac-fa56-4d2f-b6e1-a5b418026f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:48.006076 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5e9fac-fa56-4d2f-b6e1-a5b418026f3b admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:48.006266 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5e9fac-fa56-4d2f-b6e1-a5b418026f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:48.006566 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5e9fac-fa56-4d2f-b6e1-a5b418026f3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:48.007350 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5e9fac-fa56-4d2f-b6e1-a5b418026f3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:48.008985 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5e9fac-fa56-4d2f-b6e1-a5b418026f3b admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:48.022431 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1213/2447] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:50:47 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:50:48.030066 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cf4e9c-f078-45cb-8a12-1f17334d61fc admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:48.036133 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:48.041741 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cf4e9c-f078-45cb-8a12-1f17334d61fc admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:48.048514 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783a1ca0-4f41-41e4-9459-0d0961146939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:48.050189 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783a1ca0-4f41-41e4-9459-0d0961146939 admin admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:48.050468 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783a1ca0-4f41-41e4-9459-0d0961146939 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:48.050742 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783a1ca0-4f41-41e4-9459-0d0961146939 admin admin] PATH_INFO: `/v3/projects/b68d5e10933147df870b269dfc7a3932/users/3d8fa87088714a218dd04f86ee890175/roles/486144ad1bcd417e86acdbc1d14327bf` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:48.054356 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783a1ca0-4f41-41e4-9459-0d0961146939 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b68d5e10933147df870b269dfc7a3932, user_id=3d8fa87088714a218dd04f86ee890175, role_id=486144ad1bcd417e86acdbc1d14327bf)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:48.061870 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783a1ca0-4f41-41e4-9459-0d0961146939 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:48.068348 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1235/2448] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:50:48 2022] PUT /identity/v3/projects/b68d5e10933147df870b269dfc7a3932/users/3d8fa87088714a218dd04f86ee890175/roles/486144ad1bcd417e86acdbc1d14327bf => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:48.078321 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bdaf83-a8e7-457b-911a-2a604f410517 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:48.078613 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bdaf83-a8e7-457b-911a-2a604f410517 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:48.078953 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bdaf83-a8e7-457b-911a-2a604f410517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:48.107969 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-e7bdaf83-a8e7-457b-911a-2a604f410517 None None] MFA Rules not processed for user `3d8fa87088714a218dd04f86ee890175`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:48.109307 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-e7bdaf83-a8e7-457b-911a-2a604f410517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:48.190372 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1214/2449] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:50:48 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:50:48.202193 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783a1ca0-4f41-41e4-9459-0d0961146939 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:48.204104 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-783a1ca0-4f41-41e4-9459-0d0961146939 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:48.271867 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783a1ca0-4f41-41e4-9459-0d0961146939 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:48.276260 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935f1b91-8933-49c6-8eed-47a6b9460896 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36639dce4f97412f89d71367f650b901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:48.276876 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935f1b91-8933-49c6-8eed-47a6b9460896 service cinder] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:48.277165 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935f1b91-8933-49c6-8eed-47a6b9460896 service cinder] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:48.277470 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935f1b91-8933-49c6-8eed-47a6b9460896 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:48.289232 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935f1b91-8933-49c6-8eed-47a6b9460896 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:48.290692 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935f1b91-8933-49c6-8eed-47a6b9460896 service cinder] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:48.300782 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1236/2450] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:50:48 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:50:48.463983 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bdaf83-a8e7-457b-911a-2a604f410517 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:48.532779 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bdaf83-a8e7-457b-911a-2a604f410517 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:48.539640 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a934e2-f527-4dca-b1ea-51c37cdc36bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:48.540278 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a934e2-f527-4dca-b1ea-51c37cdc36bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:48.540560 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a934e2-f527-4dca-b1ea-51c37cdc36bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:48.540836 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a934e2-f527-4dca-b1ea-51c37cdc36bd admin admin] PATH_INFO: `/v3/users/85168dcd649f46b8a176400531e28057` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:48.542014 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a934e2-f527-4dca-b1ea-51c37cdc36bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=85168dcd649f46b8a176400531e28057)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:48.543949 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a934e2-f527-4dca-b1ea-51c37cdc36bd admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:48.743594 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:48.750616 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-67a934e2-f527-4dca-b1ea-51c37cdc36bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85168dcd649f46b8a176400531e28057'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:48.752467 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-67a934e2-f527-4dca-b1ea-51c37cdc36bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85168dcd649f46b8a176400531e28057'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:48.752604 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-67a934e2-f527-4dca-b1ea-51c37cdc36bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85168dcd649f46b8a176400531e28057'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:48.762746 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-67a934e2-f527-4dca-b1ea-51c37cdc36bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85168dcd649f46b8a176400531e28057'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:48.839797 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-67a934e2-f527-4dca-b1ea-51c37cdc36bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85168dcd649f46b8a176400531e28057'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:48.867363 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1215/2451] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:50:48 2022] DELETE /identity/v3/users/85168dcd649f46b8a176400531e28057 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:48.868874 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17345168-1eee-4fbd-85cd-eed6dc23d5d6 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:48.869480 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17345168-1eee-4fbd-85cd-eed6dc23d5d6 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:48.869782 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17345168-1eee-4fbd-85cd-eed6dc23d5d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:48.890899 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-17345168-1eee-4fbd-85cd-eed6dc23d5d6 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:48.892125 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-17345168-1eee-4fbd-85cd-eed6dc23d5d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:48.961558 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1237/2452] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:50:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:50:48.971526 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a934e2-f527-4dca-b1ea-51c37cdc36bd admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:48.973648 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-67a934e2-f527-4dca-b1ea-51c37cdc36bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:49.031151 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a934e2-f527-4dca-b1ea-51c37cdc36bd admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:49.034853 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b15f9e-207e-4b5d-ab06-65769a7a59dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:49.036250 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b15f9e-207e-4b5d-ab06-65769a7a59dc service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:49.036645 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b15f9e-207e-4b5d-ab06-65769a7a59dc service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:49.036846 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b15f9e-207e-4b5d-ab06-65769a7a59dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:49.045933 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b15f9e-207e-4b5d-ab06-65769a7a59dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:49.047406 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b15f9e-207e-4b5d-ab06-65769a7a59dc service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:49.056894 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1216/2453] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:50:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:50:49.488660 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17345168-1eee-4fbd-85cd-eed6dc23d5d6 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:49.490122 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-17345168-1eee-4fbd-85cd-eed6dc23d5d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:49.502524 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17345168-1eee-4fbd-85cd-eed6dc23d5d6 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:49.518120 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b2e898-405b-4fcb-a895-c691524d190d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:49.518774 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b2e898-405b-4fcb-a895-c691524d190d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:49.520513 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b2e898-405b-4fcb-a895-c691524d190d admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:49.520513 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b2e898-405b-4fcb-a895-c691524d190d admin admin] PATH_INFO: `/v3/projects/90fd5a3905e34554a71eedec8ef8d362` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:49.521974 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b2e898-405b-4fcb-a895-c691524d190d admin admin] RBAC: Authorizing `identity:delete_project(project_id=90fd5a3905e34554a71eedec8ef8d362)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:49.524049 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b2e898-405b-4fcb-a895-c691524d190d admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:49.647496 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-82b2e898-405b-4fcb-a895-c691524d190d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90fd5a3905e34554a71eedec8ef8d362', 'request_id': 'req-82b2e898-405b-4fcb-a895-c691524d190d'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:49.765028 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-82b2e898-405b-4fcb-a895-c691524d190d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90fd5a3905e34554a71eedec8ef8d362', 'request_id': 'req-82b2e898-405b-4fcb-a895-c691524d190d'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:49.775646 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-82b2e898-405b-4fcb-a895-c691524d190d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90fd5a3905e34554a71eedec8ef8d362', 'request_id': 'req-82b2e898-405b-4fcb-a895-c691524d190d'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:49.787167 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-82b2e898-405b-4fcb-a895-c691524d190d admin admin] The token cache is being invalidate because project 90fd5a3905e34554a71eedec8ef8d362 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81723) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:50:49.787465 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-82b2e898-405b-4fcb-a895-c691524d190d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:49.788121 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-82b2e898-405b-4fcb-a895-c691524d190d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:49.789681 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1238/2454] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:50:49 2022] DELETE /identity/v3/projects/90fd5a3905e34554a71eedec8ef8d362 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:49.793800 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b15f9e-207e-4b5d-ab06-65769a7a59dc service neutron] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:49.804641 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b15f9e-207e-4b5d-ab06-65769a7a59dc service neutron] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:49.863624 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e921e0-471b-46e3-881d-328b6ea76d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:49.865257 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e921e0-471b-46e3-881d-328b6ea76d40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:49.865556 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e921e0-471b-46e3-881d-328b6ea76d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:49.865828 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e921e0-471b-46e3-881d-328b6ea76d40 admin admin] PATH_INFO: `/v3/users/3d8fa87088714a218dd04f86ee890175` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:49.867026 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e921e0-471b-46e3-881d-328b6ea76d40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d8fa87088714a218dd04f86ee890175)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:49.868999 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e921e0-471b-46e3-881d-328b6ea76d40 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:50.055060 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-44e921e0-471b-46e3-881d-328b6ea76d40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d8fa87088714a218dd04f86ee890175'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:50.055719 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-44e921e0-471b-46e3-881d-328b6ea76d40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d8fa87088714a218dd04f86ee890175'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:50.056390 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-44e921e0-471b-46e3-881d-328b6ea76d40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d8fa87088714a218dd04f86ee890175'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:50.065020 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-44e921e0-471b-46e3-881d-328b6ea76d40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d8fa87088714a218dd04f86ee890175'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:50.069675 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:50.132771 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-44e921e0-471b-46e3-881d-328b6ea76d40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d8fa87088714a218dd04f86ee890175'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:50.150590 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1217/2455] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:50:49 2022] DELETE /identity/v3/users/3d8fa87088714a218dd04f86ee890175 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:50.644322 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b2e898-405b-4fcb-a895-c691524d190d admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:50.649215 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-82b2e898-405b-4fcb-a895-c691524d190d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:50.673966 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b2e898-405b-4fcb-a895-c691524d190d admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:50.715935 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:50.723226 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e921e0-471b-46e3-881d-328b6ea76d40 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:50.725327 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-44e921e0-471b-46e3-881d-328b6ea76d40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:50.791672 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9655bac-5d15-4484-a805-595899034952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:50.792307 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9655bac-5d15-4484-a805-595899034952 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:50.792593 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9655bac-5d15-4484-a805-595899034952 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:50.792867 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9655bac-5d15-4484-a805-595899034952 admin admin] PATH_INFO: `/v3/projects/b68d5e10933147df870b269dfc7a3932` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:50.794499 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9655bac-5d15-4484-a805-595899034952 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b68d5e10933147df870b269dfc7a3932)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:50.796401 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9655bac-5d15-4484-a805-595899034952 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:50.809626 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e921e0-471b-46e3-881d-328b6ea76d40 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:50.825380 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1de866-f7e2-4450-9223-ac532b6c2e44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:50.825380 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1de866-f7e2-4450-9223-ac532b6c2e44 service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:50.825380 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1de866-f7e2-4450-9223-ac532b6c2e44 service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:50.825380 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1de866-f7e2-4450-9223-ac532b6c2e44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:50.825839 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-5c1de866-f7e2-4450-9223-ac532b6c2e44 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:50.891123 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1de866-f7e2-4450-9223-ac532b6c2e44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:50.892579 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1de866-f7e2-4450-9223-ac532b6c2e44 service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:50.903034 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1218/2456] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:50:50 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:50:50.972728 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-e9655bac-5d15-4484-a805-595899034952 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b68d5e10933147df870b269dfc7a3932', 'request_id': 'req-e9655bac-5d15-4484-a805-595899034952'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:51.075170 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-e9655bac-5d15-4484-a805-595899034952 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b68d5e10933147df870b269dfc7a3932', 'request_id': 'req-e9655bac-5d15-4484-a805-595899034952'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:51.089678 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-e9655bac-5d15-4484-a805-595899034952 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b68d5e10933147df870b269dfc7a3932', 'request_id': 'req-e9655bac-5d15-4484-a805-595899034952'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:51.106414 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1de866-f7e2-4450-9223-ac532b6c2e44 service neutron] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:51.108128 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-5c1de866-f7e2-4450-9223-ac532b6c2e44 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:51.110785 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-e9655bac-5d15-4484-a805-595899034952 admin admin] The token cache is being invalidate because project b68d5e10933147df870b269dfc7a3932 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81723) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:50:51.111165 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-e9655bac-5d15-4484-a805-595899034952 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:51.111946 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-e9655bac-5d15-4484-a805-595899034952 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:51.117591 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1239/2457] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:50:50 2022] DELETE /identity/v3/projects/b68d5e10933147df870b269dfc7a3932 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:51.128905 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0539557d-632e-4517-82c3-5da664ca0246 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:51.128905 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0539557d-632e-4517-82c3-5da664ca0246 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:51.128905 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0539557d-632e-4517-82c3-5da664ca0246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:51.150532 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-0539557d-632e-4517-82c3-5da664ca0246 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:51.152830 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-0539557d-632e-4517-82c3-5da664ca0246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:51.209079 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1de866-f7e2-4450-9223-ac532b6c2e44 service neutron] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:51.235983 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:51.244649 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1240/2458] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:50:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:50:51.253688 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48ddf9b-ddc5-480d-85b3-96996555873f None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:51.254210 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48ddf9b-ddc5-480d-85b3-96996555873f None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:51.254586 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48ddf9b-ddc5-480d-85b3-96996555873f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:51.277449 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-a48ddf9b-ddc5-480d-85b3-96996555873f None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:51.279700 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-a48ddf9b-ddc5-480d-85b3-96996555873f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:51.292755 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1241/2459] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:50:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:50:51.293156 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae4ae80-5297-4794-9346-590058fd4b4b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4f49c1b08c460f8ba6c451c9c92c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:51.294014 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae4ae80-5297-4794-9346-590058fd4b4b service nova] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:51.294456 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae4ae80-5297-4794-9346-590058fd4b4b service nova] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:51.294895 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae4ae80-5297-4794-9346-590058fd4b4b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:51.296815 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48ddf9b-ddc5-480d-85b3-96996555873f None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:51.297236 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-5ae4ae80-5297-4794-9346-590058fd4b4b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:51.308485 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48ddf9b-ddc5-480d-85b3-96996555873f None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:51.313617 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faac3e3-7f9b-4576-a530-6403932bc841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:51.314368 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faac3e3-7f9b-4576-a530-6403932bc841 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:51.314783 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faac3e3-7f9b-4576-a530-6403932bc841 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:51.315324 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faac3e3-7f9b-4576-a530-6403932bc841 admin admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:51.316085 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faac3e3-7f9b-4576-a530-6403932bc841 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:50:51.316516 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faac3e3-7f9b-4576-a530-6403932bc841 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:51.318833 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faac3e3-7f9b-4576-a530-6403932bc841 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:51.332935 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1242/2460] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 24 06:50:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:50:51.337459 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faac3e3-7f9b-4576-a530-6403932bc841 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:51.351848 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faac3e3-7f9b-4576-a530-6403932bc841 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:51.356898 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12430c78-0475-4401-9603-f883f72673c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:51.358336 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12430c78-0475-4401-9603-f883f72673c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:51.358725 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12430c78-0475-4401-9603-f883f72673c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:51.359101 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12430c78-0475-4401-9603-f883f72673c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:51.360109 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12430c78-0475-4401-9603-f883f72673c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:51.362040 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12430c78-0475-4401-9603-f883f72673c9 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:51.391438 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae4ae80-5297-4794-9346-590058fd4b4b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:51.393035 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae4ae80-5297-4794-9346-590058fd4b4b service nova] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:51.413260 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1219/2461] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:50:51 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:50:51.450192 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1243/2462] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:50:51 2022] POST /identity/v3/projects => generated 383 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:50:51.455169 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae4ae80-5297-4794-9346-590058fd4b4b service nova] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:51.465503 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae4ae80-5297-4794-9346-590058fd4b4b service nova] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:51.530696 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ab7fad-c641-4fa0-aea0-1ba81c9383e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:51.531324 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ab7fad-c641-4fa0-aea0-1ba81c9383e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:51.531604 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ab7fad-c641-4fa0-aea0-1ba81c9383e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:51.531878 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ab7fad-c641-4fa0-aea0-1ba81c9383e3 admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:51.532719 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ab7fad-c641-4fa0-aea0-1ba81c9383e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:51.534465 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ab7fad-c641-4fa0-aea0-1ba81c9383e3 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:51.674429 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1220/2463] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:50:51 2022] POST /identity/v3/users => generated 292 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:50:51.675742 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12430c78-0475-4401-9603-f883f72673c9 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:51.682823 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:51.686749 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12430c78-0475-4401-9603-f883f72673c9 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:51.691367 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41de529a-8ca2-4748-95d4-7f035dfc1d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:51.691972 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41de529a-8ca2-4748-95d4-7f035dfc1d02 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:51.692254 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41de529a-8ca2-4748-95d4-7f035dfc1d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:51.692701 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41de529a-8ca2-4748-95d4-7f035dfc1d02 admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:51.693700 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41de529a-8ca2-4748-95d4-7f035dfc1d02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:51.695319 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41de529a-8ca2-4748-95d4-7f035dfc1d02 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:51.711064 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1244/2464] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:50:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:50:51.712638 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ab7fad-c641-4fa0-aea0-1ba81c9383e3 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:51.723435 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ab7fad-c641-4fa0-aea0-1ba81c9383e3 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:51.729105 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4688edb0-66a2-480c-adb0-15a11644aa7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:51.729809 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4688edb0-66a2-480c-adb0-15a11644aa7d admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:51.730154 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4688edb0-66a2-480c-adb0-15a11644aa7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:51.730550 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4688edb0-66a2-480c-adb0-15a11644aa7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:51.731255 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4688edb0-66a2-480c-adb0-15a11644aa7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:51.732918 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4688edb0-66a2-480c-adb0-15a11644aa7d admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:51.743506 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1221/2465] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:50:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:50:51.747691 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41de529a-8ca2-4748-95d4-7f035dfc1d02 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:51.758779 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41de529a-8ca2-4748-95d4-7f035dfc1d02 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:51.763366 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1696b036-79c2-4efd-aa45-ee36f5d117f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:51.764027 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1696b036-79c2-4efd-aa45-ee36f5d117f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:51.764309 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1696b036-79c2-4efd-aa45-ee36f5d117f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:51.764584 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1696b036-79c2-4efd-aa45-ee36f5d117f2 admin admin] PATH_INFO: `/v3/projects/339a70f1faf64ade9ae2230b4523f083/users/98ca0f5f5fe04acfbd4b5a041ae65b98/roles/e1f9b167c51c4a388cbc9594db8a465d` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:51.780748 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1696b036-79c2-4efd-aa45-ee36f5d117f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=339a70f1faf64ade9ae2230b4523f083, user_id=98ca0f5f5fe04acfbd4b5a041ae65b98, role_id=e1f9b167c51c4a388cbc9594db8a465d)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:51.783452 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1696b036-79c2-4efd-aa45-ee36f5d117f2 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:51.883101 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1245/2466] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:50:51 2022] PUT /identity/v3/projects/339a70f1faf64ade9ae2230b4523f083/users/98ca0f5f5fe04acfbd4b5a041ae65b98/roles/e1f9b167c51c4a388cbc9594db8a465d => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:51.894829 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d1425b-2ac8-4a54-82e6-2e4d4de15d66 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:51.907743 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d1425b-2ac8-4a54-82e6-2e4d4de15d66 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:51.907743 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d1425b-2ac8-4a54-82e6-2e4d4de15d66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:51.923617 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:51.926584 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-78d1425b-2ac8-4a54-82e6-2e4d4de15d66 None None] MFA Rules not processed for user `98ca0f5f5fe04acfbd4b5a041ae65b98`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:51.928464 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-78d1425b-2ac8-4a54-82e6-2e4d4de15d66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:51.998108 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1222/2467] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:50:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:50:52.002431 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1696b036-79c2-4efd-aa45-ee36f5d117f2 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:52.074456 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1696b036-79c2-4efd-aa45-ee36f5d117f2 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:52.079360 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d45772e-45f2-4157-88ab-869784e30833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:52.080038 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d45772e-45f2-4157-88ab-869784e30833 admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:52.080317 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d45772e-45f2-4157-88ab-869784e30833 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:52.080668 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d45772e-45f2-4157-88ab-869784e30833 admin admin] PATH_INFO: `/v3/projects` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:52.081329 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d45772e-45f2-4157-88ab-869784e30833 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:52.084059 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d45772e-45f2-4157-88ab-869784e30833 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:52.178074 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1246/2468] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:50:52 2022] POST /identity/v3/projects => generated 383 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:50:52.182396 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d1425b-2ac8-4a54-82e6-2e4d4de15d66 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:52.194502 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d1425b-2ac8-4a54-82e6-2e4d4de15d66 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:52.258761 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b9b651-bc05-4f07-8d2b-3a46e545998f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:52.259420 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b9b651-bc05-4f07-8d2b-3a46e545998f admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:52.259702 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b9b651-bc05-4f07-8d2b-3a46e545998f admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:52.259978 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b9b651-bc05-4f07-8d2b-3a46e545998f admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:52.260628 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b9b651-bc05-4f07-8d2b-3a46e545998f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:52.262714 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b9b651-bc05-4f07-8d2b-3a46e545998f admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:52.359507 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1223/2469] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:50:52 2022] POST /identity/v3/users => generated 298 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:50:52.363612 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d45772e-45f2-4157-88ab-869784e30833 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:52.374558 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d45772e-45f2-4157-88ab-869784e30833 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:52.379368 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4756b8-5006-41d4-b510-6c379b26a4eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:52.379994 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4756b8-5006-41d4-b510-6c379b26a4eb admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:52.381452 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4756b8-5006-41d4-b510-6c379b26a4eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:52.381452 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4756b8-5006-41d4-b510-6c379b26a4eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:52.381452 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4756b8-5006-41d4-b510-6c379b26a4eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:52.382971 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4756b8-5006-41d4-b510-6c379b26a4eb admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:52.394149 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1247/2470] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:50:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:50:52.399232 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b9b651-bc05-4f07-8d2b-3a46e545998f admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:52.412871 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b9b651-bc05-4f07-8d2b-3a46e545998f admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:52.417857 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4eab3b-7aa4-42db-8b5a-dce164ca6578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:52.418534 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4eab3b-7aa4-42db-8b5a-dce164ca6578 admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:52.418814 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4eab3b-7aa4-42db-8b5a-dce164ca6578 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:52.419086 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4eab3b-7aa4-42db-8b5a-dce164ca6578 admin admin] PATH_INFO: `/v3/projects/dc147261871347f3ada76c2f5e56f827/users/a3cf83560d1b411ba5acfbd1775782e1/roles/486144ad1bcd417e86acdbc1d14327bf` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:52.435674 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4eab3b-7aa4-42db-8b5a-dce164ca6578 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc147261871347f3ada76c2f5e56f827, user_id=a3cf83560d1b411ba5acfbd1775782e1, role_id=486144ad1bcd417e86acdbc1d14327bf)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:52.438451 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4eab3b-7aa4-42db-8b5a-dce164ca6578 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:52.556891 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1224/2471] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:50:52 2022] PUT /identity/v3/projects/dc147261871347f3ada76c2f5e56f827/users/a3cf83560d1b411ba5acfbd1775782e1/roles/486144ad1bcd417e86acdbc1d14327bf => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:52.561236 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4756b8-5006-41d4-b510-6c379b26a4eb admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:52.572666 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4756b8-5006-41d4-b510-6c379b26a4eb admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:52.658856 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829aacf7-8530-4db3-beb7-fa27bfee1166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:52.659585 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829aacf7-8530-4db3-beb7-fa27bfee1166 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:52.659884 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829aacf7-8530-4db3-beb7-fa27bfee1166 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:52.660162 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829aacf7-8530-4db3-beb7-fa27bfee1166 admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:52.660819 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829aacf7-8530-4db3-beb7-fa27bfee1166 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:52.662978 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829aacf7-8530-4db3-beb7-fa27bfee1166 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:52.676381 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1248/2472] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:50:52 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:50:52.680419 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4eab3b-7aa4-42db-8b5a-dce164ca6578 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:52.690882 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4eab3b-7aa4-42db-8b5a-dce164ca6578 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:52.696283 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1231e190-a729-4fa1-99c5-fc81d2414335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:52.697028 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1231e190-a729-4fa1-99c5-fc81d2414335 admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:52.697404 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1231e190-a729-4fa1-99c5-fc81d2414335 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:52.702421 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1231e190-a729-4fa1-99c5-fc81d2414335 admin admin] PATH_INFO: `/v3/projects/dc147261871347f3ada76c2f5e56f827/users/a3cf83560d1b411ba5acfbd1775782e1/roles/486144ad1bcd417e86acdbc1d14327bf` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:52.702421 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1231e190-a729-4fa1-99c5-fc81d2414335 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc147261871347f3ada76c2f5e56f827, user_id=a3cf83560d1b411ba5acfbd1775782e1, role_id=486144ad1bcd417e86acdbc1d14327bf)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:52.703081 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1231e190-a729-4fa1-99c5-fc81d2414335 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:52.715467 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1225/2473] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:50:52 2022] PUT /identity/v3/projects/dc147261871347f3ada76c2f5e56f827/users/a3cf83560d1b411ba5acfbd1775782e1/roles/486144ad1bcd417e86acdbc1d14327bf => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:52.726560 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eebdf9-76ce-4ef9-a446-5662008d1f59 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:52.730769 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eebdf9-76ce-4ef9-a446-5662008d1f59 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:52.730769 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eebdf9-76ce-4ef9-a446-5662008d1f59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:52.757981 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-71eebdf9-76ce-4ef9-a446-5662008d1f59 None None] MFA Rules not processed for user `a3cf83560d1b411ba5acfbd1775782e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:52.759513 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-71eebdf9-76ce-4ef9-a446-5662008d1f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:52.798687 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:52.842049 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1249/2474] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:50:52 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:50:52.854075 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1231e190-a729-4fa1-99c5-fc81d2414335 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:52.855793 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-1231e190-a729-4fa1-99c5-fc81d2414335 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:52.926451 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1231e190-a729-4fa1-99c5-fc81d2414335 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:52.931128 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f775d6f3-93f5-42a5-a9f8-f7c5a50e21a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36639dce4f97412f89d71367f650b901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:52.931756 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f775d6f3-93f5-42a5-a9f8-f7c5a50e21a6 service cinder] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:52.932039 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f775d6f3-93f5-42a5-a9f8-f7c5a50e21a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:52.932317 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f775d6f3-93f5-42a5-a9f8-f7c5a50e21a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:52.942673 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f775d6f3-93f5-42a5-a9f8-f7c5a50e21a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:52.944279 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f775d6f3-93f5-42a5-a9f8-f7c5a50e21a6 service cinder] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:52.948672 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:52.956574 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1226/2475] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:50:52 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:50:53.198859 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eebdf9-76ce-4ef9-a446-5662008d1f59 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:53.200716 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-71eebdf9-76ce-4ef9-a446-5662008d1f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:53.215561 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eebdf9-76ce-4ef9-a446-5662008d1f59 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:53.219574 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf7a084-8434-4779-93c9-bff3be14a055 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36639dce4f97412f89d71367f650b901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:53.220186 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf7a084-8434-4779-93c9-bff3be14a055 service cinder] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:53.220533 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf7a084-8434-4779-93c9-bff3be14a055 service cinder] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:53.220881 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf7a084-8434-4779-93c9-bff3be14a055 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:53.302388 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf7a084-8434-4779-93c9-bff3be14a055 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:53.302388 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf7a084-8434-4779-93c9-bff3be14a055 service cinder] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:53.315499 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1250/2476] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:50:53 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:50:53.450994 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367acdeb-e1b6-4815-8319-255ff5e259d4 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:53.451517 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367acdeb-e1b6-4815-8319-255ff5e259d4 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:53.451517 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367acdeb-e1b6-4815-8319-255ff5e259d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:53.480781 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-367acdeb-e1b6-4815-8319-255ff5e259d4 None None] MFA Rules not processed for user `ea70acd744994250a2905b5dd7163dfb`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:53.480781 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-367acdeb-e1b6-4815-8319-255ff5e259d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:53.576537 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1227/2477] 10.222.0.47 () {58 vars in 967 bytes} [Wed Aug 24 06:50:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:50:53.592317 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf7a084-8434-4779-93c9-bff3be14a055 service cinder] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:53.594758 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-6cf7a084-8434-4779-93c9-bff3be14a055 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:53.692158 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf7a084-8434-4779-93c9-bff3be14a055 service cinder] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:53.700323 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb094f8f-314a-4b60-8ef4-3298c8a5b0dd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02f1cbaf2cd6427585f298c9d58c319a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:53.701724 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb094f8f-314a-4b60-8ef4-3298c8a5b0dd service swift] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:53.702296 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb094f8f-314a-4b60-8ef4-3298c8a5b0dd service swift] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:53.702805 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb094f8f-314a-4b60-8ef4-3298c8a5b0dd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:53.720293 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb094f8f-314a-4b60-8ef4-3298c8a5b0dd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:53.720293 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb094f8f-314a-4b60-8ef4-3298c8a5b0dd service swift] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:53.730567 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1251/2478] 10.222.0.47 () {60 vars in 1384 bytes} [Wed Aug 24 06:50:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 06:50:53.918447 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367acdeb-e1b6-4815-8319-255ff5e259d4 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:54.050558 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367acdeb-e1b6-4815-8319-255ff5e259d4 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:54.056940 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f834afe5-d1c4-46a6-929f-ca18898a60f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:54.057796 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f834afe5-d1c4-46a6-929f-ca18898a60f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:54.058092 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f834afe5-d1c4-46a6-929f-ca18898a60f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:54.058375 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f834afe5-d1c4-46a6-929f-ca18898a60f9 admin admin] PATH_INFO: `/v3/users/98ca0f5f5fe04acfbd4b5a041ae65b98` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:54.059576 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f834afe5-d1c4-46a6-929f-ca18898a60f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98ca0f5f5fe04acfbd4b5a041ae65b98)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:54.061587 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f834afe5-d1c4-46a6-929f-ca18898a60f9 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:54.247364 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-f834afe5-d1c4-46a6-929f-ca18898a60f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98ca0f5f5fe04acfbd4b5a041ae65b98'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:54.248041 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-f834afe5-d1c4-46a6-929f-ca18898a60f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98ca0f5f5fe04acfbd4b5a041ae65b98'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:54.248644 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-f834afe5-d1c4-46a6-929f-ca18898a60f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98ca0f5f5fe04acfbd4b5a041ae65b98'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:54.256676 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-f834afe5-d1c4-46a6-929f-ca18898a60f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98ca0f5f5fe04acfbd4b5a041ae65b98'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:54.326684 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-f834afe5-d1c4-46a6-929f-ca18898a60f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98ca0f5f5fe04acfbd4b5a041ae65b98'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:54.346199 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1228/2479] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:50:53 2022] DELETE /identity/v3/users/98ca0f5f5fe04acfbd4b5a041ae65b98 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:50:54.351428 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ea9369-59e7-4e0d-92b2-33a58e33811f None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:54.351720 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ea9369-59e7-4e0d-92b2-33a58e33811f None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:54.351998 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ea9369-59e7-4e0d-92b2-33a58e33811f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:54.373383 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-26ea9369-59e7-4e0d-92b2-33a58e33811f None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:54.374619 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-26ea9369-59e7-4e0d-92b2-33a58e33811f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:54.436610 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:54.453813 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1252/2480] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:50:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:50:54.467454 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f834afe5-d1c4-46a6-929f-ca18898a60f9 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:54.469292 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-f834afe5-d1c4-46a6-929f-ca18898a60f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:54.500051 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:54.537754 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f834afe5-d1c4-46a6-929f-ca18898a60f9 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:54.541583 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf808794-6dff-47be-940a-abda687c21d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:54.543020 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf808794-6dff-47be-940a-abda687c21d1 service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:54.543020 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf808794-6dff-47be-940a-abda687c21d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:54.543020 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf808794-6dff-47be-940a-abda687c21d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:54.552440 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf808794-6dff-47be-940a-abda687c21d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:54.553951 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf808794-6dff-47be-940a-abda687c21d1 service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:54.563891 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1229/2481] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:50:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:50:54.978483 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ea9369-59e7-4e0d-92b2-33a58e33811f None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:54.979944 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-26ea9369-59e7-4e0d-92b2-33a58e33811f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:54.992785 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ea9369-59e7-4e0d-92b2-33a58e33811f None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:54.999295 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1654932-f93a-4822-a1ae-d9063d3329a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:54.999927 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1654932-f93a-4822-a1ae-d9063d3329a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:55.000235 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1654932-f93a-4822-a1ae-d9063d3329a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:55.000508 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1654932-f93a-4822-a1ae-d9063d3329a8 admin admin] PATH_INFO: `/v3/projects/339a70f1faf64ade9ae2230b4523f083` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:55.002164 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1654932-f93a-4822-a1ae-d9063d3329a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=339a70f1faf64ade9ae2230b4523f083)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:55.004078 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1654932-f93a-4822-a1ae-d9063d3329a8 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:55.133408 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-b1654932-f93a-4822-a1ae-d9063d3329a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '339a70f1faf64ade9ae2230b4523f083', 'request_id': 'req-b1654932-f93a-4822-a1ae-d9063d3329a8'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:55.229677 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-b1654932-f93a-4822-a1ae-d9063d3329a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '339a70f1faf64ade9ae2230b4523f083', 'request_id': 'req-b1654932-f93a-4822-a1ae-d9063d3329a8'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:55.237344 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-b1654932-f93a-4822-a1ae-d9063d3329a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '339a70f1faf64ade9ae2230b4523f083', 'request_id': 'req-b1654932-f93a-4822-a1ae-d9063d3329a8'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:55.257740 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-b1654932-f93a-4822-a1ae-d9063d3329a8 admin admin] The token cache is being invalidate because project 339a70f1faf64ade9ae2230b4523f083 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81723) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:50:55.258280 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-b1654932-f93a-4822-a1ae-d9063d3329a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:55.259153 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-b1654932-f93a-4822-a1ae-d9063d3329a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:55.261114 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1253/2482] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:50:54 2022] DELETE /identity/v3/projects/339a70f1faf64ade9ae2230b4523f083 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:55.264841 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf808794-6dff-47be-940a-abda687c21d1 service neutron] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:55.275768 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf808794-6dff-47be-940a-abda687c21d1 service neutron] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:55.344166 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7e84c5-3d94-46c1-81b6-9b75d9aa82ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:55.344877 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7e84c5-3d94-46c1-81b6-9b75d9aa82ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:55.345250 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7e84c5-3d94-46c1-81b6-9b75d9aa82ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:55.345592 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7e84c5-3d94-46c1-81b6-9b75d9aa82ff admin admin] PATH_INFO: `/v3/users/a3cf83560d1b411ba5acfbd1775782e1` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:55.346806 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7e84c5-3d94-46c1-81b6-9b75d9aa82ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3cf83560d1b411ba5acfbd1775782e1)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:55.349552 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7e84c5-3d94-46c1-81b6-9b75d9aa82ff admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:55.558969 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-1c7e84c5-3d94-46c1-81b6-9b75d9aa82ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3cf83560d1b411ba5acfbd1775782e1'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:55.559625 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-1c7e84c5-3d94-46c1-81b6-9b75d9aa82ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3cf83560d1b411ba5acfbd1775782e1'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:55.560300 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-1c7e84c5-3d94-46c1-81b6-9b75d9aa82ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3cf83560d1b411ba5acfbd1775782e1'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:55.568620 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-1c7e84c5-3d94-46c1-81b6-9b75d9aa82ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3cf83560d1b411ba5acfbd1775782e1'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:55.647564 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-1c7e84c5-3d94-46c1-81b6-9b75d9aa82ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3cf83560d1b411ba5acfbd1775782e1'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:55.666124 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1230/2483] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:50:55 2022] DELETE /identity/v3/users/a3cf83560d1b411ba5acfbd1775782e1 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:50:56.088670 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1654932-f93a-4822-a1ae-d9063d3329a8 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:56.090222 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-b1654932-f93a-4822-a1ae-d9063d3329a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:56.104208 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1654932-f93a-4822-a1ae-d9063d3329a8 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:56.163764 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320e9897-3114-4ead-b92e-6b2e974cf5f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:56.164394 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320e9897-3114-4ead-b92e-6b2e974cf5f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:56.164671 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320e9897-3114-4ead-b92e-6b2e974cf5f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:56.164975 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320e9897-3114-4ead-b92e-6b2e974cf5f9 admin admin] PATH_INFO: `/v3/projects/dc147261871347f3ada76c2f5e56f827` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:56.166199 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320e9897-3114-4ead-b92e-6b2e974cf5f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc147261871347f3ada76c2f5e56f827)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:56.168214 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320e9897-3114-4ead-b92e-6b2e974cf5f9 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:56.269512 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-320e9897-3114-4ead-b92e-6b2e974cf5f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc147261871347f3ada76c2f5e56f827', 'request_id': 'req-320e9897-3114-4ead-b92e-6b2e974cf5f9'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:56.364213 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-320e9897-3114-4ead-b92e-6b2e974cf5f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc147261871347f3ada76c2f5e56f827', 'request_id': 'req-320e9897-3114-4ead-b92e-6b2e974cf5f9'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:56.372320 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-320e9897-3114-4ead-b92e-6b2e974cf5f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc147261871347f3ada76c2f5e56f827', 'request_id': 'req-320e9897-3114-4ead-b92e-6b2e974cf5f9'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:56.388937 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-320e9897-3114-4ead-b92e-6b2e974cf5f9 admin admin] The token cache is being invalidate because project dc147261871347f3ada76c2f5e56f827 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81723) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:50:56.389259 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-320e9897-3114-4ead-b92e-6b2e974cf5f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:56.393995 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-320e9897-3114-4ead-b92e-6b2e974cf5f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:50:56.393995 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1254/2484] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:50:56 2022] DELETE /identity/v3/projects/dc147261871347f3ada76c2f5e56f827 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:56.398251 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f77a6fa-4c7e-40bb-b1a3-d0c97936ca81 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:56.398544 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f77a6fa-4c7e-40bb-b1a3-d0c97936ca81 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:56.398825 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f77a6fa-4c7e-40bb-b1a3-d0c97936ca81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:56.419273 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-2f77a6fa-4c7e-40bb-b1a3-d0c97936ca81 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:56.420735 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-2f77a6fa-4c7e-40bb-b1a3-d0c97936ca81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:56.491560 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:56.515365 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1231/2485] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:50:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:50:56.522837 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea0edf6-db6d-4e9a-acb6-d41405e0d045 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:56.522837 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea0edf6-db6d-4e9a-acb6-d41405e0d045 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:56.523219 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea0edf6-db6d-4e9a-acb6-d41405e0d045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:56.538520 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:56.544453 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-5ea0edf6-db6d-4e9a-acb6-d41405e0d045 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:56.545712 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-5ea0edf6-db6d-4e9a-acb6-d41405e0d045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:56.555484 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1255/2486] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:50:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:50:56.563361 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f77a6fa-4c7e-40bb-b1a3-d0c97936ca81 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:56.575796 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f77a6fa-4c7e-40bb-b1a3-d0c97936ca81 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:56.581284 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572eaa32-bf66-444d-aacf-b430127c2c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:56.582000 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572eaa32-bf66-444d-aacf-b430127c2c14 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:56.582280 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572eaa32-bf66-444d-aacf-b430127c2c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:56.582571 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572eaa32-bf66-444d-aacf-b430127c2c14 admin admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:56.583970 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572eaa32-bf66-444d-aacf-b430127c2c14 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:50:56.584333 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572eaa32-bf66-444d-aacf-b430127c2c14 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:56.585974 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572eaa32-bf66-444d-aacf-b430127c2c14 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:56.599552 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1232/2487] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 24 06:50:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:50:56.604040 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea0edf6-db6d-4e9a-acb6-d41405e0d045 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:56.613775 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea0edf6-db6d-4e9a-acb6-d41405e0d045 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:56.623142 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f51537-541a-4819-b781-8cf13f2670d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:56.623771 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f51537-541a-4819-b781-8cf13f2670d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:56.624136 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f51537-541a-4819-b781-8cf13f2670d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:56.624489 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f51537-541a-4819-b781-8cf13f2670d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:56.625291 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f51537-541a-4819-b781-8cf13f2670d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:56.627344 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f51537-541a-4819-b781-8cf13f2670d0 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:56.719519 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1256/2488] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:50:56 2022] POST /identity/v3/projects => generated 373 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:50:56.728163 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572eaa32-bf66-444d-aacf-b430127c2c14 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:56.738137 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572eaa32-bf66-444d-aacf-b430127c2c14 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:56.796013 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1d55a7-d0f4-4d0c-a8ef-293548a08d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:56.796734 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d55a7-d0f4-4d0c-a8ef-293548a08d53 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:56.797019 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d55a7-d0f4-4d0c-a8ef-293548a08d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:56.797336 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d55a7-d0f4-4d0c-a8ef-293548a08d53 admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:56.797976 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1d55a7-d0f4-4d0c-a8ef-293548a08d53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:56.799934 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1d55a7-d0f4-4d0c-a8ef-293548a08d53 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:56.924774 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1233/2489] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:50:56 2022] POST /identity/v3/users => generated 287 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:50:56.929855 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f51537-541a-4819-b781-8cf13f2670d0 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:56.940317 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f51537-541a-4819-b781-8cf13f2670d0 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:56.945825 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd9c9b7-caf7-4b17-bd95-d8f2283d9dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:56.946437 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd9c9b7-caf7-4b17-bd95-d8f2283d9dea admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:56.946719 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd9c9b7-caf7-4b17-bd95-d8f2283d9dea admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:56.946996 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd9c9b7-caf7-4b17-bd95-d8f2283d9dea admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:56.947653 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd9c9b7-caf7-4b17-bd95-d8f2283d9dea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:56.949303 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd9c9b7-caf7-4b17-bd95-d8f2283d9dea admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:56.960938 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1257/2490] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:50:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:50:56.965216 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1d55a7-d0f4-4d0c-a8ef-293548a08d53 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:56.978097 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1d55a7-d0f4-4d0c-a8ef-293548a08d53 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:56.984525 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9109a98-4853-4a65-bb2f-a01847544e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:56.991452 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9109a98-4853-4a65-bb2f-a01847544e0d admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:56.991452 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9109a98-4853-4a65-bb2f-a01847544e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:56.991452 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9109a98-4853-4a65-bb2f-a01847544e0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:56.991452 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9109a98-4853-4a65-bb2f-a01847544e0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:56.991452 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9109a98-4853-4a65-bb2f-a01847544e0d admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:57.000123 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1234/2491] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:50:56 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:50:57.004541 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd9c9b7-caf7-4b17-bd95-d8f2283d9dea admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:57.016535 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd9c9b7-caf7-4b17-bd95-d8f2283d9dea admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:57.023226 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb9b9b9-18f8-401b-ac86-5579d866aa66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:57.023988 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb9b9b9-18f8-401b-ac86-5579d866aa66 admin admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:57.024269 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb9b9b9-18f8-401b-ac86-5579d866aa66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:57.024540 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb9b9b9-18f8-401b-ac86-5579d866aa66 admin admin] PATH_INFO: `/v3/projects/e7ff705387724e169c84fc07b1c5b498/users/3dbb7cbdcea748c8876ab28ad8f895a3/roles/e1f9b167c51c4a388cbc9594db8a465d` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:57.050840 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb9b9b9-18f8-401b-ac86-5579d866aa66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7ff705387724e169c84fc07b1c5b498, user_id=3dbb7cbdcea748c8876ab28ad8f895a3, role_id=e1f9b167c51c4a388cbc9594db8a465d)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:57.056811 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb9b9b9-18f8-401b-ac86-5579d866aa66 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:57.151011 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1258/2492] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:50:57 2022] PUT /identity/v3/projects/e7ff705387724e169c84fc07b1c5b498/users/3dbb7cbdcea748c8876ab28ad8f895a3/roles/e1f9b167c51c4a388cbc9594db8a465d => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:57.157578 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5b79cb-ecbf-4d3f-ac4b-0fae820089c3 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:57.157891 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5b79cb-ecbf-4d3f-ac4b-0fae820089c3 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:57.158171 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5b79cb-ecbf-4d3f-ac4b-0fae820089c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:57.182586 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-4c5b79cb-ecbf-4d3f-ac4b-0fae820089c3 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:57.183954 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-4c5b79cb-ecbf-4d3f-ac4b-0fae820089c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:57.251440 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1235/2493] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:50:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:50:57.263293 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb9b9b9-18f8-401b-ac86-5579d866aa66 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:57.265141 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-9cb9b9b9-18f8-401b-ac86-5579d866aa66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:57.334366 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb9b9b9-18f8-401b-ac86-5579d866aa66 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:57.338424 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4985c4-e612-461c-b1e3-7d827b4ffa7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:57.339138 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4985c4-e612-461c-b1e3-7d827b4ffa7a service neutron] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:57.339495 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4985c4-e612-461c-b1e3-7d827b4ffa7a service neutron] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:57.339847 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4985c4-e612-461c-b1e3-7d827b4ffa7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:57.360961 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4985c4-e612-461c-b1e3-7d827b4ffa7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:57.362471 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4985c4-e612-461c-b1e3-7d827b4ffa7a service neutron] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:57.374383 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1259/2494] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:50:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:50:59.034586 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3945804c-8716-4f0a-a2b4-2e80a2eede66 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:59.039314 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3945804c-8716-4f0a-a2b4-2e80a2eede66 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:59.039314 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3945804c-8716-4f0a-a2b4-2e80a2eede66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:59.068233 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-3945804c-8716-4f0a-a2b4-2e80a2eede66 None None] MFA Rules not processed for user `3dbb7cbdcea748c8876ab28ad8f895a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:50:59.070100 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-3945804c-8716-4f0a-a2b4-2e80a2eede66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:50:59.122462 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:50:59.155858 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1236/2495] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:50:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:50:59.160170 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4985c4-e612-461c-b1e3-7d827b4ffa7a service neutron] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:59.173713 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4985c4-e612-461c-b1e3-7d827b4ffa7a service neutron] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:59.178195 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2885b69-3118-42a6-b8b1-beccf77ca7ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:59.178787 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2885b69-3118-42a6-b8b1-beccf77ca7ca admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:59.179067 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2885b69-3118-42a6-b8b1-beccf77ca7ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:59.179338 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2885b69-3118-42a6-b8b1-beccf77ca7ca admin admin] PATH_INFO: `/v3/projects` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:59.179963 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2885b69-3118-42a6-b8b1-beccf77ca7ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:59.181970 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2885b69-3118-42a6-b8b1-beccf77ca7ca admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:59.378212 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1260/2496] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:50:59 2022] POST /identity/v3/projects => generated 371 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:50:59.382881 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3945804c-8716-4f0a-a2b4-2e80a2eede66 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:59.393804 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3945804c-8716-4f0a-a2b4-2e80a2eede66 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:59.452080 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167f4735-bf7b-4f67-b882-dfa492b14a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:59.452710 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167f4735-bf7b-4f67-b882-dfa492b14a24 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:59.452978 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167f4735-bf7b-4f67-b882-dfa492b14a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:59.453290 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167f4735-bf7b-4f67-b882-dfa492b14a24 admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:59.453941 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167f4735-bf7b-4f67-b882-dfa492b14a24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:59.455859 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167f4735-bf7b-4f67-b882-dfa492b14a24 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:59.589874 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1237/2497] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:50:59 2022] POST /identity/v3/users => generated 292 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:50:59.595647 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2885b69-3118-42a6-b8b1-beccf77ca7ca admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:59.608210 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2885b69-3118-42a6-b8b1-beccf77ca7ca admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:59.613004 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddcbd5a-1591-41d3-a733-d238fe6d2021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:59.613750 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddcbd5a-1591-41d3-a733-d238fe6d2021 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:59.614029 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddcbd5a-1591-41d3-a733-d238fe6d2021 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:59.614302 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddcbd5a-1591-41d3-a733-d238fe6d2021 admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:59.614945 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddcbd5a-1591-41d3-a733-d238fe6d2021 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:59.617637 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddcbd5a-1591-41d3-a733-d238fe6d2021 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:59.835376 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1261/2498] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:50:59 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:50:59.835514 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167f4735-bf7b-4f67-b882-dfa492b14a24 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:59.835514 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167f4735-bf7b-4f67-b882-dfa492b14a24 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:59.835514 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a50185-6114-4c55-a416-138c55b8055b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:59.835514 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a50185-6114-4c55-a416-138c55b8055b admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:59.835900 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a50185-6114-4c55-a416-138c55b8055b admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:59.835900 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a50185-6114-4c55-a416-138c55b8055b admin admin] PATH_INFO: `/v3/projects/a17c048213b44d3fa64d972bc082469c/users/6fcfc35b845f46d69dc13fd41efcbcde/roles/486144ad1bcd417e86acdbc1d14327bf` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:59.835900 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a50185-6114-4c55-a416-138c55b8055b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a17c048213b44d3fa64d972bc082469c, user_id=6fcfc35b845f46d69dc13fd41efcbcde, role_id=486144ad1bcd417e86acdbc1d14327bf)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:59.835900 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a50185-6114-4c55-a416-138c55b8055b admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:59.883123 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1238/2499] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:50:59 2022] PUT /identity/v3/projects/a17c048213b44d3fa64d972bc082469c/users/6fcfc35b845f46d69dc13fd41efcbcde/roles/486144ad1bcd417e86acdbc1d14327bf => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:50:59.887325 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddcbd5a-1591-41d3-a733-d238fe6d2021 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:59.899557 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddcbd5a-1591-41d3-a733-d238fe6d2021 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:59.958511 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e25f1dc-25dd-41fc-b1ef-7f0b16ac75bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:59.959802 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e25f1dc-25dd-41fc-b1ef-7f0b16ac75bb admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:59.960698 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e25f1dc-25dd-41fc-b1ef-7f0b16ac75bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:59.960960 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e25f1dc-25dd-41fc-b1ef-7f0b16ac75bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:59.961640 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e25f1dc-25dd-41fc-b1ef-7f0b16ac75bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:59.963233 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e25f1dc-25dd-41fc-b1ef-7f0b16ac75bb admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:50:59.973865 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1262/2500] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:50:59 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:50:59.978142 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a50185-6114-4c55-a416-138c55b8055b admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:50:59.988717 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a50185-6114-4c55-a416-138c55b8055b admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:50:59.993328 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342ec3ad-5cfa-45eb-832b-2c493e60aa01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:50:59.994040 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342ec3ad-5cfa-45eb-832b-2c493e60aa01 admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:50:59.994321 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342ec3ad-5cfa-45eb-832b-2c493e60aa01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:50:59.994598 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342ec3ad-5cfa-45eb-832b-2c493e60aa01 admin admin] PATH_INFO: `/v3/projects/a17c048213b44d3fa64d972bc082469c/users/6fcfc35b845f46d69dc13fd41efcbcde/roles/486144ad1bcd417e86acdbc1d14327bf` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:50:59.996505 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342ec3ad-5cfa-45eb-832b-2c493e60aa01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a17c048213b44d3fa64d972bc082469c, user_id=6fcfc35b845f46d69dc13fd41efcbcde, role_id=486144ad1bcd417e86acdbc1d14327bf)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:50:59.999216 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342ec3ad-5cfa-45eb-832b-2c493e60aa01 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:00.010203 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1239/2501] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:50:59 2022] PUT /identity/v3/projects/a17c048213b44d3fa64d972bc082469c/users/6fcfc35b845f46d69dc13fd41efcbcde/roles/486144ad1bcd417e86acdbc1d14327bf => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:51:01.514449 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30626e9b-acea-49f9-ab3e-819b3ed1847e None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:01.514934 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30626e9b-acea-49f9-ab3e-819b3ed1847e None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:01.515052 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30626e9b-acea-49f9-ab3e-819b3ed1847e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:01.536268 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:01.546477 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-30626e9b-acea-49f9-ab3e-819b3ed1847e None None] MFA Rules not processed for user `6fcfc35b845f46d69dc13fd41efcbcde`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:51:01.547743 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-30626e9b-acea-49f9-ab3e-819b3ed1847e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:01.624202 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1263/2502] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:51:01 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:51:01.638058 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342ec3ad-5cfa-45eb-832b-2c493e60aa01 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:01.642321 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-342ec3ad-5cfa-45eb-832b-2c493e60aa01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:01.713717 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342ec3ad-5cfa-45eb-832b-2c493e60aa01 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:01.717594 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02be5a4-3c35-47d1-bb0f-804156fb4bbb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36639dce4f97412f89d71367f650b901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:01.718218 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02be5a4-3c35-47d1-bb0f-804156fb4bbb service cinder] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:01.718499 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02be5a4-3c35-47d1-bb0f-804156fb4bbb service cinder] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:01.718783 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02be5a4-3c35-47d1-bb0f-804156fb4bbb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:01.766266 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:01.778642 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02be5a4-3c35-47d1-bb0f-804156fb4bbb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:01.780067 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02be5a4-3c35-47d1-bb0f-804156fb4bbb service cinder] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:01.789807 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1240/2503] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:51:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:51:03.601475 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30626e9b-acea-49f9-ab3e-819b3ed1847e None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:03.612045 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30626e9b-acea-49f9-ab3e-819b3ed1847e None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:03.615752 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9052ef-0405-4882-93c3-bab0dfa03df5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36639dce4f97412f89d71367f650b901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:03.616358 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9052ef-0405-4882-93c3-bab0dfa03df5 service cinder] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:03.616645 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9052ef-0405-4882-93c3-bab0dfa03df5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:03.617020 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9052ef-0405-4882-93c3-bab0dfa03df5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:03.629953 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9052ef-0405-4882-93c3-bab0dfa03df5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:03.629953 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9052ef-0405-4882-93c3-bab0dfa03df5 service cinder] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:03.640305 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1264/2504] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:51:03 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:51:04.179332 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02be5a4-3c35-47d1-bb0f-804156fb4bbb service cinder] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:04.236774 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02be5a4-3c35-47d1-bb0f-804156fb4bbb service cinder] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:04.240467 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c704f71-a50c-416c-8dad-2ea518e64342 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:04.241073 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c704f71-a50c-416c-8dad-2ea518e64342 service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:04.241424 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c704f71-a50c-416c-8dad-2ea518e64342 service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:04.241698 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c704f71-a50c-416c-8dad-2ea518e64342 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:04.244179 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-4c704f71-a50c-416c-8dad-2ea518e64342 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:04.316011 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c704f71-a50c-416c-8dad-2ea518e64342 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:04.317717 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c704f71-a50c-416c-8dad-2ea518e64342 service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:04.328636 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1241/2505] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:51:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:51:08.053635 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9052ef-0405-4882-93c3-bab0dfa03df5 service cinder] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:08.061120 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-ae9052ef-0405-4882-93c3-bab0dfa03df5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:08.071819 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9052ef-0405-4882-93c3-bab0dfa03df5 service cinder] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:08.079243 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4b029c-2ffa-43fb-9ffe-d71e67b6f89a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:08.080015 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4b029c-2ffa-43fb-9ffe-d71e67b6f89a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:08.080336 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4b029c-2ffa-43fb-9ffe-d71e67b6f89a admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:08.080644 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4b029c-2ffa-43fb-9ffe-d71e67b6f89a admin admin] PATH_INFO: `/v3/users/8b9b29b53a7b490da29fbf6e5451c362` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:08.218955 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4b029c-2ffa-43fb-9ffe-d71e67b6f89a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b9b29b53a7b490da29fbf6e5451c362)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:08.220979 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4b029c-2ffa-43fb-9ffe-d71e67b6f89a admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:08.473783 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-8b4b029c-2ffa-43fb-9ffe-d71e67b6f89a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b9b29b53a7b490da29fbf6e5451c362'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:08.474479 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-8b4b029c-2ffa-43fb-9ffe-d71e67b6f89a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b9b29b53a7b490da29fbf6e5451c362'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:08.475083 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-8b4b029c-2ffa-43fb-9ffe-d71e67b6f89a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b9b29b53a7b490da29fbf6e5451c362'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:08.543657 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-8b4b029c-2ffa-43fb-9ffe-d71e67b6f89a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b9b29b53a7b490da29fbf6e5451c362'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:08.557723 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-8b4b029c-2ffa-43fb-9ffe-d71e67b6f89a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b9b29b53a7b490da29fbf6e5451c362'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:08.570651 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1265/2506] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:51:08 2022] DELETE /identity/v3/users/8b9b29b53a7b490da29fbf6e5451c362 => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:51:08.847725 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c704f71-a50c-416c-8dad-2ea518e64342 service neutron] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:08.858290 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c704f71-a50c-416c-8dad-2ea518e64342 service neutron] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:08.931685 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce61400-e064-415d-a519-dcf2258c49e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:08.932475 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce61400-e064-415d-a519-dcf2258c49e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:08.932760 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce61400-e064-415d-a519-dcf2258c49e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:08.933039 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce61400-e064-415d-a519-dcf2258c49e9 admin admin] PATH_INFO: `/v3/projects/096ae1367ad84c7aaa0b5e96e22e9e32` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:08.944250 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce61400-e064-415d-a519-dcf2258c49e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=096ae1367ad84c7aaa0b5e96e22e9e32)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:08.947021 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce61400-e064-415d-a519-dcf2258c49e9 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:09.020351 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:09.033603 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-9ce61400-e064-415d-a519-dcf2258c49e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '096ae1367ad84c7aaa0b5e96e22e9e32', 'request_id': 'req-9ce61400-e064-415d-a519-dcf2258c49e9'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:09.040946 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-9ce61400-e064-415d-a519-dcf2258c49e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '096ae1367ad84c7aaa0b5e96e22e9e32', 'request_id': 'req-9ce61400-e064-415d-a519-dcf2258c49e9'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:09.123623 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-9ce61400-e064-415d-a519-dcf2258c49e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '096ae1367ad84c7aaa0b5e96e22e9e32', 'request_id': 'req-9ce61400-e064-415d-a519-dcf2258c49e9'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:09.136532 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-9ce61400-e064-415d-a519-dcf2258c49e9 admin admin] The token cache is being invalidate because project 096ae1367ad84c7aaa0b5e96e22e9e32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:51:09.136826 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-9ce61400-e064-415d-a519-dcf2258c49e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:09.137492 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-9ce61400-e064-415d-a519-dcf2258c49e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:09.140426 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1242/2507] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:51:08 2022] DELETE /identity/v3/projects/096ae1367ad84c7aaa0b5e96e22e9e32 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:51:09.145154 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da9355c-13d6-497e-b809-998f67557a6f None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:09.145687 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da9355c-13d6-497e-b809-998f67557a6f None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:09.145786 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da9355c-13d6-497e-b809-998f67557a6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:09.166595 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-0da9355c-13d6-497e-b809-998f67557a6f None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:51:09.169121 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-0da9355c-13d6-497e-b809-998f67557a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:09.207433 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:09.228353 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1266/2508] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:51:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:51:09.236716 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e21b1e-a8f5-45ad-a605-2df8b9a273c7 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:09.237874 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e21b1e-a8f5-45ad-a605-2df8b9a273c7 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:09.237874 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e21b1e-a8f5-45ad-a605-2df8b9a273c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:09.257903 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-b2e21b1e-a8f5-45ad-a605-2df8b9a273c7 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:51:09.259902 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-b2e21b1e-a8f5-45ad-a605-2df8b9a273c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:09.267996 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1243/2509] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:51:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:51:09.272285 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da9355c-13d6-497e-b809-998f67557a6f None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:09.282069 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da9355c-13d6-497e-b809-998f67557a6f None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:09.286528 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad4ac38-6600-424f-b2f9-a7461e9dfdfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:09.287115 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad4ac38-6600-424f-b2f9-a7461e9dfdfb admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:09.287398 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad4ac38-6600-424f-b2f9-a7461e9dfdfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:09.287671 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad4ac38-6600-424f-b2f9-a7461e9dfdfb admin admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:09.288304 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad4ac38-6600-424f-b2f9-a7461e9dfdfb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:51:09.288597 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad4ac38-6600-424f-b2f9-a7461e9dfdfb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:09.290283 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad4ac38-6600-424f-b2f9-a7461e9dfdfb admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:09.301663 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1267/2510] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 24 06:51:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:51:09.306976 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e21b1e-a8f5-45ad-a605-2df8b9a273c7 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:09.316186 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e21b1e-a8f5-45ad-a605-2df8b9a273c7 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:09.321526 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ccd7c3-db8b-4293-b67d-e764b3b2b34c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:09.322301 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ccd7c3-db8b-4293-b67d-e764b3b2b34c admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:09.322582 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ccd7c3-db8b-4293-b67d-e764b3b2b34c admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:09.322855 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ccd7c3-db8b-4293-b67d-e764b3b2b34c admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:09.323478 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ccd7c3-db8b-4293-b67d-e764b3b2b34c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:09.325399 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ccd7c3-db8b-4293-b67d-e764b3b2b34c admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:09.425460 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1244/2511] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:51:09 2022] POST /identity/v3/projects => generated 391 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:51:09.429586 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad4ac38-6600-424f-b2f9-a7461e9dfdfb admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:09.440954 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad4ac38-6600-424f-b2f9-a7461e9dfdfb admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:09.500687 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a08488-fb9b-4c6a-ac30-fe385b81cefd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:09.501334 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a08488-fb9b-4c6a-ac30-fe385b81cefd admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:09.501619 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a08488-fb9b-4c6a-ac30-fe385b81cefd admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:09.501890 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a08488-fb9b-4c6a-ac30-fe385b81cefd admin admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:09.502545 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a08488-fb9b-4c6a-ac30-fe385b81cefd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:09.504459 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a08488-fb9b-4c6a-ac30-fe385b81cefd admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:09.601671 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1268/2512] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:51:09 2022] POST /identity/v3/users => generated 296 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:51:09.606448 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ccd7c3-db8b-4293-b67d-e764b3b2b34c admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:09.616859 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ccd7c3-db8b-4293-b67d-e764b3b2b34c admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:09.621906 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5d2f82-a723-43f1-856a-32f254722e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:09.623152 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5d2f82-a723-43f1-856a-32f254722e63 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:09.623468 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5d2f82-a723-43f1-856a-32f254722e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:09.623746 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5d2f82-a723-43f1-856a-32f254722e63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:09.624401 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5d2f82-a723-43f1-856a-32f254722e63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:09.626099 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5d2f82-a723-43f1-856a-32f254722e63 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:09.636574 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1245/2513] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:51:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:51:09.640803 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a08488-fb9b-4c6a-ac30-fe385b81cefd admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:09.650953 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a08488-fb9b-4c6a-ac30-fe385b81cefd admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:09.656248 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47605138-d792-49ce-b17d-7ecbdee557f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:09.656878 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47605138-d792-49ce-b17d-7ecbdee557f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:09.657150 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47605138-d792-49ce-b17d-7ecbdee557f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:09.657455 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47605138-d792-49ce-b17d-7ecbdee557f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:09.658107 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47605138-d792-49ce-b17d-7ecbdee557f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:09.659706 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47605138-d792-49ce-b17d-7ecbdee557f4 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:09.670656 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1269/2514] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:51:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:51:09.674877 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5d2f82-a723-43f1-856a-32f254722e63 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:09.686047 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5d2f82-a723-43f1-856a-32f254722e63 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:09.690566 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d10cca-3eef-4964-a846-ee13240a3cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:09.691213 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d10cca-3eef-4964-a846-ee13240a3cab admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:09.691495 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d10cca-3eef-4964-a846-ee13240a3cab admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:09.691771 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d10cca-3eef-4964-a846-ee13240a3cab admin admin] PATH_INFO: `/v3/projects/9a37121a013241258666543676ed6b81/users/2f2fa04f301941909f3211b50a66177e/roles/e1f9b167c51c4a388cbc9594db8a465d` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:09.706596 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d10cca-3eef-4964-a846-ee13240a3cab admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a37121a013241258666543676ed6b81, user_id=2f2fa04f301941909f3211b50a66177e, role_id=e1f9b167c51c4a388cbc9594db8a465d)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:09.709282 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d10cca-3eef-4964-a846-ee13240a3cab admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:09.779708 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1246/2515] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:51:09 2022] PUT /identity/v3/projects/9a37121a013241258666543676ed6b81/users/2f2fa04f301941909f3211b50a66177e/roles/e1f9b167c51c4a388cbc9594db8a465d => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:51:09.792109 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d80e77-724c-4924-aa6f-a1d6631775d6 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:09.798133 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d80e77-724c-4924-aa6f-a1d6631775d6 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:09.798133 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d80e77-724c-4924-aa6f-a1d6631775d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:09.822825 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-67d80e77-724c-4924-aa6f-a1d6631775d6 None None] MFA Rules not processed for user `2f2fa04f301941909f3211b50a66177e`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:51:09.824026 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-67d80e77-724c-4924-aa6f-a1d6631775d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:09.890743 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1270/2516] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:51:09 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:51:09.895291 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d10cca-3eef-4964-a846-ee13240a3cab admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:09.958642 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d10cca-3eef-4964-a846-ee13240a3cab admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:09.964118 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62525607-6b91-4869-9e75-896949c50693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:09.964760 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62525607-6b91-4869-9e75-896949c50693 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:09.965046 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62525607-6b91-4869-9e75-896949c50693 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:09.965428 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62525607-6b91-4869-9e75-896949c50693 admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:09.966055 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62525607-6b91-4869-9e75-896949c50693 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:09.968227 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62525607-6b91-4869-9e75-896949c50693 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:10.106426 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1247/2517] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:51:09 2022] POST /identity/v3/projects => generated 391 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:51:10.110727 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d80e77-724c-4924-aa6f-a1d6631775d6 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:10.122479 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d80e77-724c-4924-aa6f-a1d6631775d6 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:10.157646 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:10.181520 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccb13ac-289c-4007-a6b5-61ebec6e38a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:10.182156 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccb13ac-289c-4007-a6b5-61ebec6e38a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:10.182544 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccb13ac-289c-4007-a6b5-61ebec6e38a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:10.182823 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccb13ac-289c-4007-a6b5-61ebec6e38a3 admin admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:10.183455 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccb13ac-289c-4007-a6b5-61ebec6e38a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:10.185424 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccb13ac-289c-4007-a6b5-61ebec6e38a3 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:10.292046 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1271/2518] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:51:10 2022] POST /identity/v3/users => generated 302 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:51:10.296071 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62525607-6b91-4869-9e75-896949c50693 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:10.307536 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62525607-6b91-4869-9e75-896949c50693 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:10.312492 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892b78f4-aecf-4f62-b0a4-33bfc252f63a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:10.313138 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892b78f4-aecf-4f62-b0a4-33bfc252f63a admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:10.313451 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892b78f4-aecf-4f62-b0a4-33bfc252f63a admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:10.313729 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892b78f4-aecf-4f62-b0a4-33bfc252f63a admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:10.314377 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892b78f4-aecf-4f62-b0a4-33bfc252f63a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:10.315972 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892b78f4-aecf-4f62-b0a4-33bfc252f63a admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:10.327499 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1248/2519] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:51:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:51:10.331854 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccb13ac-289c-4007-a6b5-61ebec6e38a3 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:10.343387 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccb13ac-289c-4007-a6b5-61ebec6e38a3 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:10.348495 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41698e6-b133-40a1-a17f-19485daef41a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:10.353032 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41698e6-b133-40a1-a17f-19485daef41a admin admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:10.353032 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41698e6-b133-40a1-a17f-19485daef41a admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:10.353032 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41698e6-b133-40a1-a17f-19485daef41a admin admin] PATH_INFO: `/v3/projects/a941347faf85483aaafa24e5334183f4/users/ddfccc990018436f80b4e3c15be74b3c/roles/486144ad1bcd417e86acdbc1d14327bf` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:10.364334 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41698e6-b133-40a1-a17f-19485daef41a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a941347faf85483aaafa24e5334183f4, user_id=ddfccc990018436f80b4e3c15be74b3c, role_id=486144ad1bcd417e86acdbc1d14327bf)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:10.367025 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41698e6-b133-40a1-a17f-19485daef41a admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:10.447471 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1272/2520] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:51:10 2022] PUT /identity/v3/projects/a941347faf85483aaafa24e5334183f4/users/ddfccc990018436f80b4e3c15be74b3c/roles/486144ad1bcd417e86acdbc1d14327bf => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:51:10.451543 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892b78f4-aecf-4f62-b0a4-33bfc252f63a admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:10.462394 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892b78f4-aecf-4f62-b0a4-33bfc252f63a admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:10.488672 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:10.520025 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa02178-031f-4b53-baa3-b350f5a0b288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:10.520628 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa02178-031f-4b53-baa3-b350f5a0b288 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:10.520915 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa02178-031f-4b53-baa3-b350f5a0b288 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:10.521197 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa02178-031f-4b53-baa3-b350f5a0b288 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:10.521903 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa02178-031f-4b53-baa3-b350f5a0b288 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:10.524903 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa02178-031f-4b53-baa3-b350f5a0b288 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:10.534222 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1249/2521] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:51:10 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:51:10.538483 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41698e6-b133-40a1-a17f-19485daef41a admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:10.548918 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41698e6-b133-40a1-a17f-19485daef41a admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:10.553487 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb0b4de-a527-42b6-a8c1-507a6e75badd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:10.554193 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb0b4de-a527-42b6-a8c1-507a6e75badd admin admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:10.554470 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb0b4de-a527-42b6-a8c1-507a6e75badd admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:10.554743 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb0b4de-a527-42b6-a8c1-507a6e75badd admin admin] PATH_INFO: `/v3/projects/a941347faf85483aaafa24e5334183f4/users/ddfccc990018436f80b4e3c15be74b3c/roles/486144ad1bcd417e86acdbc1d14327bf` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:10.557327 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb0b4de-a527-42b6-a8c1-507a6e75badd admin admin] RBAC: Authorizing `identity:create_grant(project_id=a941347faf85483aaafa24e5334183f4, user_id=ddfccc990018436f80b4e3c15be74b3c, role_id=486144ad1bcd417e86acdbc1d14327bf)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:10.560055 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb0b4de-a527-42b6-a8c1-507a6e75badd admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:10.570367 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1273/2522] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:51:10 2022] PUT /identity/v3/projects/a941347faf85483aaafa24e5334183f4/users/ddfccc990018436f80b4e3c15be74b3c/roles/486144ad1bcd417e86acdbc1d14327bf => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:51:10.587668 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8dd451-8731-464d-9284-227667fc0b9c None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:10.587984 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8dd451-8731-464d-9284-227667fc0b9c None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:10.588269 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8dd451-8731-464d-9284-227667fc0b9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:10.617548 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-7f8dd451-8731-464d-9284-227667fc0b9c None None] MFA Rules not processed for user `ddfccc990018436f80b4e3c15be74b3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:51:10.619166 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-7f8dd451-8731-464d-9284-227667fc0b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:10.689557 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1250/2523] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:51:10 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:51:10.707246 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb0b4de-a527-42b6-a8c1-507a6e75badd admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:10.708931 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-efb0b4de-a527-42b6-a8c1-507a6e75badd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:10.766908 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb0b4de-a527-42b6-a8c1-507a6e75badd admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:10.770616 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2108ecd5-226e-49ed-8c9f-1921dbf22bb4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36639dce4f97412f89d71367f650b901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:10.771220 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2108ecd5-226e-49ed-8c9f-1921dbf22bb4 service cinder] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:10.771501 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2108ecd5-226e-49ed-8c9f-1921dbf22bb4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:10.771776 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2108ecd5-226e-49ed-8c9f-1921dbf22bb4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:10.782364 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2108ecd5-226e-49ed-8c9f-1921dbf22bb4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:10.783824 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2108ecd5-226e-49ed-8c9f-1921dbf22bb4 service cinder] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:10.794567 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1274/2524] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:51:10 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:51:10.840765 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8dd451-8731-464d-9284-227667fc0b9c None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:10.842488 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-7f8dd451-8731-464d-9284-227667fc0b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:10.855059 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8dd451-8731-464d-9284-227667fc0b9c None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:10.858702 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab02348b-d07d-4695-ba7e-8d35158ef6da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36639dce4f97412f89d71367f650b901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:10.859302 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab02348b-d07d-4695-ba7e-8d35158ef6da service cinder] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:10.859585 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab02348b-d07d-4695-ba7e-8d35158ef6da service cinder] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:10.859860 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab02348b-d07d-4695-ba7e-8d35158ef6da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:10.918411 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab02348b-d07d-4695-ba7e-8d35158ef6da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:10.919854 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab02348b-d07d-4695-ba7e-8d35158ef6da service cinder] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:10.929462 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1251/2525] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:51:10 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:51:15.848888 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2108ecd5-226e-49ed-8c9f-1921dbf22bb4 service cinder] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:15.916026 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2108ecd5-226e-49ed-8c9f-1921dbf22bb4 service cinder] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:15.921053 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95213dea-9f48-42aa-bd3f-2b69a6b26198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:15.921713 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95213dea-9f48-42aa-bd3f-2b69a6b26198 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:15.921993 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95213dea-9f48-42aa-bd3f-2b69a6b26198 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:15.922361 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95213dea-9f48-42aa-bd3f-2b69a6b26198 admin admin] PATH_INFO: `/v3/users/2f2fa04f301941909f3211b50a66177e` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:15.923496 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95213dea-9f48-42aa-bd3f-2b69a6b26198 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f2fa04f301941909f3211b50a66177e)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:15.925476 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95213dea-9f48-42aa-bd3f-2b69a6b26198 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:16.081313 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:16.091488 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-95213dea-9f48-42aa-bd3f-2b69a6b26198 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f2fa04f301941909f3211b50a66177e'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:16.092441 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-95213dea-9f48-42aa-bd3f-2b69a6b26198 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f2fa04f301941909f3211b50a66177e'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:16.093067 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-95213dea-9f48-42aa-bd3f-2b69a6b26198 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f2fa04f301941909f3211b50a66177e'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:16.229921 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-95213dea-9f48-42aa-bd3f-2b69a6b26198 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f2fa04f301941909f3211b50a66177e'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:16.245815 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-95213dea-9f48-42aa-bd3f-2b69a6b26198 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f2fa04f301941909f3211b50a66177e'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:16.258243 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1275/2526] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:51:15 2022] DELETE /identity/v3/users/2f2fa04f301941909f3211b50a66177e => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:51:16.263009 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68df657-7d50-458c-ae07-f4f83900b604 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:16.263516 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68df657-7d50-458c-ae07-f4f83900b604 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:16.263616 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68df657-7d50-458c-ae07-f4f83900b604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:16.285028 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-e68df657-7d50-458c-ae07-f4f83900b604 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:51:16.286284 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-e68df657-7d50-458c-ae07-f4f83900b604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:16.332059 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:16.350826 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1252/2527] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:51:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:51:16.360862 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95213dea-9f48-42aa-bd3f-2b69a6b26198 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:16.362287 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-95213dea-9f48-42aa-bd3f-2b69a6b26198 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:16.419943 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95213dea-9f48-42aa-bd3f-2b69a6b26198 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:16.423837 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c1335a-e810-48f9-9eec-c4dae04929e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:16.424446 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c1335a-e810-48f9-9eec-c4dae04929e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:16.424736 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c1335a-e810-48f9-9eec-c4dae04929e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:16.425120 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c1335a-e810-48f9-9eec-c4dae04929e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:16.439527 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c1335a-e810-48f9-9eec-c4dae04929e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:16.439527 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c1335a-e810-48f9-9eec-c4dae04929e5 service neutron] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:16.449446 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1276/2528] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:51:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:51:16.882262 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68df657-7d50-458c-ae07-f4f83900b604 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:16.884580 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-e68df657-7d50-458c-ae07-f4f83900b604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:16.898422 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68df657-7d50-458c-ae07-f4f83900b604 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:16.904678 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9609ae5-b39c-4e24-998f-228bebf0d484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:16.906889 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9609ae5-b39c-4e24-998f-228bebf0d484 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:16.906889 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9609ae5-b39c-4e24-998f-228bebf0d484 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:16.906889 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9609ae5-b39c-4e24-998f-228bebf0d484 admin admin] PATH_INFO: `/v3/projects/9a37121a013241258666543676ed6b81` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:16.907178 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9609ae5-b39c-4e24-998f-228bebf0d484 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a37121a013241258666543676ed6b81)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:16.908990 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9609ae5-b39c-4e24-998f-228bebf0d484 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:17.032997 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-a9609ae5-b39c-4e24-998f-228bebf0d484 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a37121a013241258666543676ed6b81', 'request_id': 'req-a9609ae5-b39c-4e24-998f-228bebf0d484'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:17.039711 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-a9609ae5-b39c-4e24-998f-228bebf0d484 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a37121a013241258666543676ed6b81', 'request_id': 'req-a9609ae5-b39c-4e24-998f-228bebf0d484'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:17.143717 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-a9609ae5-b39c-4e24-998f-228bebf0d484 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a37121a013241258666543676ed6b81', 'request_id': 'req-a9609ae5-b39c-4e24-998f-228bebf0d484'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:17.157311 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-a9609ae5-b39c-4e24-998f-228bebf0d484 admin admin] The token cache is being invalidate because project 9a37121a013241258666543676ed6b81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:51:17.157607 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-a9609ae5-b39c-4e24-998f-228bebf0d484 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:17.158246 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-a9609ae5-b39c-4e24-998f-228bebf0d484 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:17.159725 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1253/2529] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:51:16 2022] DELETE /identity/v3/projects/9a37121a013241258666543676ed6b81 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:51:17.163756 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c1335a-e810-48f9-9eec-c4dae04929e5 service neutron] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:17.173496 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c1335a-e810-48f9-9eec-c4dae04929e5 service neutron] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:17.234930 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4925e82-9e28-4db1-b048-989831a77fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:17.235563 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4925e82-9e28-4db1-b048-989831a77fc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:17.235861 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4925e82-9e28-4db1-b048-989831a77fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:17.236142 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4925e82-9e28-4db1-b048-989831a77fc9 admin admin] PATH_INFO: `/v3/users/ddfccc990018436f80b4e3c15be74b3c` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:17.237345 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4925e82-9e28-4db1-b048-989831a77fc9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ddfccc990018436f80b4e3c15be74b3c)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:17.239268 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4925e82-9e28-4db1-b048-989831a77fc9 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:17.415774 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-e4925e82-9e28-4db1-b048-989831a77fc9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ddfccc990018436f80b4e3c15be74b3c'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:17.416433 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-e4925e82-9e28-4db1-b048-989831a77fc9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ddfccc990018436f80b4e3c15be74b3c'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:17.417049 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-e4925e82-9e28-4db1-b048-989831a77fc9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ddfccc990018436f80b4e3c15be74b3c'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:17.508834 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-e4925e82-9e28-4db1-b048-989831a77fc9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ddfccc990018436f80b4e3c15be74b3c'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:17.511075 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:17.521499 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-e4925e82-9e28-4db1-b048-989831a77fc9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ddfccc990018436f80b4e3c15be74b3c'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:17.535069 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1277/2530] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:51:17 2022] DELETE /identity/v3/users/ddfccc990018436f80b4e3c15be74b3c => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:51:17.931332 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9609ae5-b39c-4e24-998f-228bebf0d484 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:17.935349 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-a9609ae5-b39c-4e24-998f-228bebf0d484 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:17.947843 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9609ae5-b39c-4e24-998f-228bebf0d484 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:18.011067 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bff6087-ad1c-4fb4-adb8-8104b088823f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:18.011711 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bff6087-ad1c-4fb4-adb8-8104b088823f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:18.011990 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bff6087-ad1c-4fb4-adb8-8104b088823f admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:18.012282 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bff6087-ad1c-4fb4-adb8-8104b088823f admin admin] PATH_INFO: `/v3/projects/a941347faf85483aaafa24e5334183f4` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:18.013443 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bff6087-ad1c-4fb4-adb8-8104b088823f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a941347faf85483aaafa24e5334183f4)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:18.015352 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bff6087-ad1c-4fb4-adb8-8104b088823f admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:18.166880 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-3bff6087-ad1c-4fb4-adb8-8104b088823f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a941347faf85483aaafa24e5334183f4', 'request_id': 'req-3bff6087-ad1c-4fb4-adb8-8104b088823f'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:18.172501 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-3bff6087-ad1c-4fb4-adb8-8104b088823f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a941347faf85483aaafa24e5334183f4', 'request_id': 'req-3bff6087-ad1c-4fb4-adb8-8104b088823f'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:18.275114 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-3bff6087-ad1c-4fb4-adb8-8104b088823f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a941347faf85483aaafa24e5334183f4', 'request_id': 'req-3bff6087-ad1c-4fb4-adb8-8104b088823f'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:18.288856 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-3bff6087-ad1c-4fb4-adb8-8104b088823f admin admin] The token cache is being invalidate because project a941347faf85483aaafa24e5334183f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:51:18.289242 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-3bff6087-ad1c-4fb4-adb8-8104b088823f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:18.289937 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-3bff6087-ad1c-4fb4-adb8-8104b088823f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:18.291462 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1254/2531] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:51:17 2022] DELETE /identity/v3/projects/a941347faf85483aaafa24e5334183f4 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:51:18.297463 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f15c97-a21c-4b56-bb8a-e00d93fa8f88 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:18.297944 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f15c97-a21c-4b56-bb8a-e00d93fa8f88 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:18.298173 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f15c97-a21c-4b56-bb8a-e00d93fa8f88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:18.319328 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-d9f15c97-a21c-4b56-bb8a-e00d93fa8f88 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:51:18.321465 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-d9f15c97-a21c-4b56-bb8a-e00d93fa8f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:18.385956 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1278/2532] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:51:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:51:18.393957 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf8292f-45bb-476e-9de1-28229e2c142b None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:18.394377 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf8292f-45bb-476e-9de1-28229e2c142b None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:18.394754 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf8292f-45bb-476e-9de1-28229e2c142b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:18.402713 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:18.427359 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-bdf8292f-45bb-476e-9de1-28229e2c142b None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:51:18.427359 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-bdf8292f-45bb-476e-9de1-28229e2c142b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:18.427359 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1255/2533] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:51:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:51:18.430715 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f15c97-a21c-4b56-bb8a-e00d93fa8f88 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:18.441018 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f15c97-a21c-4b56-bb8a-e00d93fa8f88 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:18.447954 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618e9bbe-6c56-4928-bab1-08a2a6f29253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:18.448566 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618e9bbe-6c56-4928-bab1-08a2a6f29253 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:18.448843 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618e9bbe-6c56-4928-bab1-08a2a6f29253 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:18.449159 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618e9bbe-6c56-4928-bab1-08a2a6f29253 admin admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:18.449834 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618e9bbe-6c56-4928-bab1-08a2a6f29253 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:51:18.450128 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618e9bbe-6c56-4928-bab1-08a2a6f29253 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:18.451710 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618e9bbe-6c56-4928-bab1-08a2a6f29253 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:18.463941 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1279/2534] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 24 06:51:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:51:18.468996 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf8292f-45bb-476e-9de1-28229e2c142b None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:18.478851 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf8292f-45bb-476e-9de1-28229e2c142b None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:18.483520 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301f54aa-c32e-4815-9abd-3df0f5410185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:18.484128 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f54aa-c32e-4815-9abd-3df0f5410185 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:18.484409 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f54aa-c32e-4815-9abd-3df0f5410185 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:18.484682 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f54aa-c32e-4815-9abd-3df0f5410185 admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:18.485342 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301f54aa-c32e-4815-9abd-3df0f5410185 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:18.487269 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301f54aa-c32e-4815-9abd-3df0f5410185 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:18.607742 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1256/2535] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:51:18 2022] POST /identity/v3/projects => generated 361 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:51:18.611089 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618e9bbe-6c56-4928-bab1-08a2a6f29253 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:18.622764 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618e9bbe-6c56-4928-bab1-08a2a6f29253 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:18.684707 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9751eb23-23de-4a84-b153-634b66f7ee3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:18.685341 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9751eb23-23de-4a84-b153-634b66f7ee3e admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:18.685620 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9751eb23-23de-4a84-b153-634b66f7ee3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:18.685893 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9751eb23-23de-4a84-b153-634b66f7ee3e admin admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:18.686517 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9751eb23-23de-4a84-b153-634b66f7ee3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:18.688465 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9751eb23-23de-4a84-b153-634b66f7ee3e admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:18.774575 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1280/2536] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:51:18 2022] POST /identity/v3/users => generated 281 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:51:18.775853 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301f54aa-c32e-4815-9abd-3df0f5410185 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:18.786675 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301f54aa-c32e-4815-9abd-3df0f5410185 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:18.792675 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce2428c-e6dc-4507-a60c-e7bfc8438287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:18.793331 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce2428c-e6dc-4507-a60c-e7bfc8438287 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:18.793641 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce2428c-e6dc-4507-a60c-e7bfc8438287 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:18.793918 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce2428c-e6dc-4507-a60c-e7bfc8438287 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:18.794559 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce2428c-e6dc-4507-a60c-e7bfc8438287 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:18.796217 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce2428c-e6dc-4507-a60c-e7bfc8438287 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:18.807206 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1257/2537] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:51:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:51:18.811456 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9751eb23-23de-4a84-b153-634b66f7ee3e admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:18.821931 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9751eb23-23de-4a84-b153-634b66f7ee3e admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:18.827018 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c579125d-f382-4305-9879-ef2ad4ee98da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:18.827623 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c579125d-f382-4305-9879-ef2ad4ee98da admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:18.827902 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c579125d-f382-4305-9879-ef2ad4ee98da admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:18.828197 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c579125d-f382-4305-9879-ef2ad4ee98da admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:18.828813 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c579125d-f382-4305-9879-ef2ad4ee98da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:18.830423 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c579125d-f382-4305-9879-ef2ad4ee98da admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:18.842355 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1281/2538] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:51:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:51:18.847522 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce2428c-e6dc-4507-a60c-e7bfc8438287 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:18.858854 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce2428c-e6dc-4507-a60c-e7bfc8438287 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:18.863614 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f7a11c-8174-4904-ae7e-cdf7e3523aab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:18.871465 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f7a11c-8174-4904-ae7e-cdf7e3523aab admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:18.871465 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f7a11c-8174-4904-ae7e-cdf7e3523aab admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:18.871465 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f7a11c-8174-4904-ae7e-cdf7e3523aab admin admin] PATH_INFO: `/v3/projects/eadc15dcfb84469998aa05e9f9ac6741/users/3e396f9ec9be4ecb986d62d223c2546c/roles/e1f9b167c51c4a388cbc9594db8a465d` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:18.879802 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f7a11c-8174-4904-ae7e-cdf7e3523aab admin admin] RBAC: Authorizing `identity:create_grant(project_id=eadc15dcfb84469998aa05e9f9ac6741, user_id=3e396f9ec9be4ecb986d62d223c2546c, role_id=e1f9b167c51c4a388cbc9594db8a465d)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:18.882608 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f7a11c-8174-4904-ae7e-cdf7e3523aab admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:18.955106 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1258/2539] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:51:18 2022] PUT /identity/v3/projects/eadc15dcfb84469998aa05e9f9ac6741/users/3e396f9ec9be4ecb986d62d223c2546c/roles/e1f9b167c51c4a388cbc9594db8a465d => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:51:18.967742 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a254e8-b307-4dd4-ba3e-8b1a01ecbee7 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:18.968072 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a254e8-b307-4dd4-ba3e-8b1a01ecbee7 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:18.968350 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a254e8-b307-4dd4-ba3e-8b1a01ecbee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:18.998117 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-f2a254e8-b307-4dd4-ba3e-8b1a01ecbee7 None None] MFA Rules not processed for user `3e396f9ec9be4ecb986d62d223c2546c`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:51:18.999438 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-f2a254e8-b307-4dd4-ba3e-8b1a01ecbee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:19.029513 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:19.066309 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1282/2540] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:51:18 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:51:19.111074 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f7a11c-8174-4904-ae7e-cdf7e3523aab admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:19.112777 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-89f7a11c-8174-4904-ae7e-cdf7e3523aab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:19.172829 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f7a11c-8174-4904-ae7e-cdf7e3523aab admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:19.176785 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfb3547-607c-4340-9708-aa7c8d344bcc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36639dce4f97412f89d71367f650b901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:19.177440 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfb3547-607c-4340-9708-aa7c8d344bcc service cinder] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:19.177721 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfb3547-607c-4340-9708-aa7c8d344bcc service cinder] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:19.177992 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfb3547-607c-4340-9708-aa7c8d344bcc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:19.187413 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfb3547-607c-4340-9708-aa7c8d344bcc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:19.188842 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfb3547-607c-4340-9708-aa7c8d344bcc service cinder] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:19.198673 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1259/2541] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:51:19 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:51:19.214934 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a254e8-b307-4dd4-ba3e-8b1a01ecbee7 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:19.281653 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a254e8-b307-4dd4-ba3e-8b1a01ecbee7 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:19.286285 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d292eabb-9136-45c1-bd7f-8fcf8e748db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:19.286911 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d292eabb-9136-45c1-bd7f-8fcf8e748db4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:19.287197 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d292eabb-9136-45c1-bd7f-8fcf8e748db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:19.287474 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d292eabb-9136-45c1-bd7f-8fcf8e748db4 admin admin] PATH_INFO: `/v3/users/3e396f9ec9be4ecb986d62d223c2546c` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:19.288619 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d292eabb-9136-45c1-bd7f-8fcf8e748db4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e396f9ec9be4ecb986d62d223c2546c)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:19.290772 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d292eabb-9136-45c1-bd7f-8fcf8e748db4 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:19.480007 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-d292eabb-9136-45c1-bd7f-8fcf8e748db4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e396f9ec9be4ecb986d62d223c2546c'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:19.480670 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-d292eabb-9136-45c1-bd7f-8fcf8e748db4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e396f9ec9be4ecb986d62d223c2546c'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:19.481276 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-d292eabb-9136-45c1-bd7f-8fcf8e748db4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e396f9ec9be4ecb986d62d223c2546c'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:19.560580 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-d292eabb-9136-45c1-bd7f-8fcf8e748db4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e396f9ec9be4ecb986d62d223c2546c'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:19.574258 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-d292eabb-9136-45c1-bd7f-8fcf8e748db4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e396f9ec9be4ecb986d62d223c2546c'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:19.586803 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1283/2542] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:51:19 2022] DELETE /identity/v3/users/3e396f9ec9be4ecb986d62d223c2546c => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:51:19.591482 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ccfde-840e-48a4-8584-9e7dea840cd7 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:19.591762 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ccfde-840e-48a4-8584-9e7dea840cd7 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:19.592287 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ccfde-840e-48a4-8584-9e7dea840cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:19.611488 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-c46ccfde-840e-48a4-8584-9e7dea840cd7 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:51:19.612692 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-c46ccfde-840e-48a4-8584-9e7dea840cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:19.674956 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1260/2543] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:51:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:51:19.685055 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d292eabb-9136-45c1-bd7f-8fcf8e748db4 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:19.686493 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-d292eabb-9136-45c1-bd7f-8fcf8e748db4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:19.745714 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:19.749770 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d292eabb-9136-45c1-bd7f-8fcf8e748db4 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:19.753839 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98ac0f4-c54d-431f-99e0-deb7b20b6488 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:19.754515 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98ac0f4-c54d-431f-99e0-deb7b20b6488 service neutron] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:19.754802 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98ac0f4-c54d-431f-99e0-deb7b20b6488 service neutron] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:19.755073 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98ac0f4-c54d-431f-99e0-deb7b20b6488 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:19.766159 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98ac0f4-c54d-431f-99e0-deb7b20b6488 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:19.768002 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98ac0f4-c54d-431f-99e0-deb7b20b6488 service neutron] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:19.777815 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1284/2544] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:51:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:51:20.224659 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46ccfde-840e-48a4-8584-9e7dea840cd7 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:20.226209 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-c46ccfde-840e-48a4-8584-9e7dea840cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:20.240197 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46ccfde-840e-48a4-8584-9e7dea840cd7 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:20.245616 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f07474-1d08-42ee-9fb7-695620422993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:20.246260 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f07474-1d08-42ee-9fb7-695620422993 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:20.246545 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f07474-1d08-42ee-9fb7-695620422993 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:20.246819 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f07474-1d08-42ee-9fb7-695620422993 admin admin] PATH_INFO: `/v3/projects/eadc15dcfb84469998aa05e9f9ac6741` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:20.248143 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f07474-1d08-42ee-9fb7-695620422993 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eadc15dcfb84469998aa05e9f9ac6741)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:20.250103 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f07474-1d08-42ee-9fb7-695620422993 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:20.349902 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:20.359345 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-d9f07474-1d08-42ee-9fb7-695620422993 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eadc15dcfb84469998aa05e9f9ac6741', 'request_id': 'req-d9f07474-1d08-42ee-9fb7-695620422993'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:20.364988 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-d9f07474-1d08-42ee-9fb7-695620422993 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eadc15dcfb84469998aa05e9f9ac6741', 'request_id': 'req-d9f07474-1d08-42ee-9fb7-695620422993'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:20.486343 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-d9f07474-1d08-42ee-9fb7-695620422993 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eadc15dcfb84469998aa05e9f9ac6741', 'request_id': 'req-d9f07474-1d08-42ee-9fb7-695620422993'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:20.501640 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-d9f07474-1d08-42ee-9fb7-695620422993 admin admin] The token cache is being invalidate because project eadc15dcfb84469998aa05e9f9ac6741 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:51:20.501640 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-d9f07474-1d08-42ee-9fb7-695620422993 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:20.501640 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-d9f07474-1d08-42ee-9fb7-695620422993 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:20.501640 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1261/2545] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:51:20 2022] DELETE /identity/v3/projects/eadc15dcfb84469998aa05e9f9ac6741 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:51:20.507550 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c3928f-d435-4f95-898f-d9823a213157 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:20.508107 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c3928f-d435-4f95-898f-d9823a213157 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:20.508752 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c3928f-d435-4f95-898f-d9823a213157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:20.534996 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-e4c3928f-d435-4f95-898f-d9823a213157 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:51:20.536257 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-e4c3928f-d435-4f95-898f-d9823a213157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:20.612257 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1285/2546] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:51:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:51:20.620306 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae61bb11-7a0e-4840-a5f6-d20bd10b8a98 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:20.620629 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae61bb11-7a0e-4840-a5f6-d20bd10b8a98 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:20.620865 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae61bb11-7a0e-4840-a5f6-d20bd10b8a98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:20.652232 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-ae61bb11-7a0e-4840-a5f6-d20bd10b8a98 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:51:20.653573 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-ae61bb11-7a0e-4840-a5f6-d20bd10b8a98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:20.661672 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1262/2547] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:51:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:51:20.667967 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c3928f-d435-4f95-898f-d9823a213157 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:20.678776 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c3928f-d435-4f95-898f-d9823a213157 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:20.684233 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255dc593-9768-48ea-bf66-b3612c8924a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:20.684838 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255dc593-9768-48ea-bf66-b3612c8924a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:20.685115 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255dc593-9768-48ea-bf66-b3612c8924a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:20.685426 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255dc593-9768-48ea-bf66-b3612c8924a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:20.686079 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255dc593-9768-48ea-bf66-b3612c8924a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:51:20.686381 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255dc593-9768-48ea-bf66-b3612c8924a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:20.687963 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255dc593-9768-48ea-bf66-b3612c8924a2 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:20.703233 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1286/2548] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 24 06:51:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:51:20.717360 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae61bb11-7a0e-4840-a5f6-d20bd10b8a98 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:20.727426 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae61bb11-7a0e-4840-a5f6-d20bd10b8a98 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:20.732448 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398d2807-1ed7-4679-bd8b-6d37917865fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:20.733060 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398d2807-1ed7-4679-bd8b-6d37917865fe admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:20.733403 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398d2807-1ed7-4679-bd8b-6d37917865fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:20.733644 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398d2807-1ed7-4679-bd8b-6d37917865fe admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:20.734273 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398d2807-1ed7-4679-bd8b-6d37917865fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:20.736753 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398d2807-1ed7-4679-bd8b-6d37917865fe admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:20.848142 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1263/2549] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:51:20 2022] POST /identity/v3/projects => generated 383 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:51:20.853975 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255dc593-9768-48ea-bf66-b3612c8924a2 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:20.868846 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255dc593-9768-48ea-bf66-b3612c8924a2 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:20.931652 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c73daf-f570-42d9-8701-368496b82def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:20.932267 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c73daf-f570-42d9-8701-368496b82def admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:20.932547 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c73daf-f570-42d9-8701-368496b82def admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:20.932820 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c73daf-f570-42d9-8701-368496b82def admin admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:20.933493 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c73daf-f570-42d9-8701-368496b82def admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:20.935407 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c73daf-f570-42d9-8701-368496b82def admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:21.038851 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1287/2550] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:51:20 2022] POST /identity/v3/users => generated 292 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:51:21.040471 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398d2807-1ed7-4679-bd8b-6d37917865fe admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:21.051837 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398d2807-1ed7-4679-bd8b-6d37917865fe admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:21.057979 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94510c38-6d8d-4bc5-839b-674f7a597b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:21.058602 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94510c38-6d8d-4bc5-839b-674f7a597b55 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:21.058886 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94510c38-6d8d-4bc5-839b-674f7a597b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:21.059164 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94510c38-6d8d-4bc5-839b-674f7a597b55 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:21.059813 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94510c38-6d8d-4bc5-839b-674f7a597b55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:21.061468 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94510c38-6d8d-4bc5-839b-674f7a597b55 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:21.073125 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1264/2551] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:51:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:51:21.080862 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c73daf-f570-42d9-8701-368496b82def admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:21.090592 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c73daf-f570-42d9-8701-368496b82def admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:21.095743 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c916a4bb-8dbb-4712-94af-73219a62d6e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:21.096350 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c916a4bb-8dbb-4712-94af-73219a62d6e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:21.096628 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c916a4bb-8dbb-4712-94af-73219a62d6e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:21.096902 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c916a4bb-8dbb-4712-94af-73219a62d6e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:21.097576 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c916a4bb-8dbb-4712-94af-73219a62d6e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:21.099227 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c916a4bb-8dbb-4712-94af-73219a62d6e0 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:21.110433 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1288/2552] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:51:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:51:21.118517 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94510c38-6d8d-4bc5-839b-674f7a597b55 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:21.129329 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94510c38-6d8d-4bc5-839b-674f7a597b55 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:21.135239 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf4677a-16ab-4e0b-b148-f17ba7d312af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:21.136451 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf4677a-16ab-4e0b-b148-f17ba7d312af admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:21.136451 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf4677a-16ab-4e0b-b148-f17ba7d312af admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:21.136731 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf4677a-16ab-4e0b-b148-f17ba7d312af admin admin] PATH_INFO: `/v3/projects/89bc91d11de14892b9598d5ced6f3848/users/81c51fc4d4c74fde8d5f73def83713e4/roles/e1f9b167c51c4a388cbc9594db8a465d` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:21.158363 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf4677a-16ab-4e0b-b148-f17ba7d312af admin admin] RBAC: Authorizing `identity:create_grant(project_id=89bc91d11de14892b9598d5ced6f3848, user_id=81c51fc4d4c74fde8d5f73def83713e4, role_id=e1f9b167c51c4a388cbc9594db8a465d)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:21.163924 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf4677a-16ab-4e0b-b148-f17ba7d312af admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:21.286687 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1265/2553] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:51:21 2022] PUT /identity/v3/projects/89bc91d11de14892b9598d5ced6f3848/users/81c51fc4d4c74fde8d5f73def83713e4/roles/e1f9b167c51c4a388cbc9594db8a465d => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:51:21.298338 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d67981-a21b-4672-a151-d4030f948c94 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:21.298543 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d67981-a21b-4672-a151-d4030f948c94 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:21.298812 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d67981-a21b-4672-a151-d4030f948c94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:21.328024 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-c0d67981-a21b-4672-a151-d4030f948c94 None None] MFA Rules not processed for user `81c51fc4d4c74fde8d5f73def83713e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:51:21.329268 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-c0d67981-a21b-4672-a151-d4030f948c94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:21.382150 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:21.403758 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1289/2554] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:51:21 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:51:21.416108 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf4677a-16ab-4e0b-b148-f17ba7d312af admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:21.418339 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-ecf4677a-16ab-4e0b-b148-f17ba7d312af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:21.477209 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf4677a-16ab-4e0b-b148-f17ba7d312af admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:21.481084 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d265863-60ff-42d1-a1e6-85cf6cf0806a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36639dce4f97412f89d71367f650b901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:21.481730 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d265863-60ff-42d1-a1e6-85cf6cf0806a service cinder] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:21.482005 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d265863-60ff-42d1-a1e6-85cf6cf0806a service cinder] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:21.482274 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d265863-60ff-42d1-a1e6-85cf6cf0806a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:21.491931 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d265863-60ff-42d1-a1e6-85cf6cf0806a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:21.494589 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d265863-60ff-42d1-a1e6-85cf6cf0806a service cinder] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:21.503899 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1266/2555] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:51:21 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:51:22.046134 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d67981-a21b-4672-a151-d4030f948c94 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:22.047707 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-c0d67981-a21b-4672-a151-d4030f948c94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:22.122120 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d67981-a21b-4672-a151-d4030f948c94 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:22.126836 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7647c5-357f-469f-90ae-30fc73bf1338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:22.127554 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7647c5-357f-469f-90ae-30fc73bf1338 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:22.127835 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7647c5-357f-469f-90ae-30fc73bf1338 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:22.128104 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7647c5-357f-469f-90ae-30fc73bf1338 admin admin] PATH_INFO: `/v3/users/3dbb7cbdcea748c8876ab28ad8f895a3` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:22.129267 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7647c5-357f-469f-90ae-30fc73bf1338 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3dbb7cbdcea748c8876ab28ad8f895a3)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:22.131178 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7647c5-357f-469f-90ae-30fc73bf1338 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:22.370382 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-db7647c5-357f-469f-90ae-30fc73bf1338 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3dbb7cbdcea748c8876ab28ad8f895a3'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:22.370957 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-db7647c5-357f-469f-90ae-30fc73bf1338 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3dbb7cbdcea748c8876ab28ad8f895a3'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:22.371594 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-db7647c5-357f-469f-90ae-30fc73bf1338 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3dbb7cbdcea748c8876ab28ad8f895a3'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:22.457720 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-db7647c5-357f-469f-90ae-30fc73bf1338 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3dbb7cbdcea748c8876ab28ad8f895a3'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:22.471592 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-db7647c5-357f-469f-90ae-30fc73bf1338 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3dbb7cbdcea748c8876ab28ad8f895a3'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:22.484352 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1290/2556] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:51:22 2022] DELETE /identity/v3/users/3dbb7cbdcea748c8876ab28ad8f895a3 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:51:22.733855 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d265863-60ff-42d1-a1e6-85cf6cf0806a service cinder] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:22.745668 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d265863-60ff-42d1-a1e6-85cf6cf0806a service cinder] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:22.806169 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce19500-b322-45e6-bf24-c4fc34a59a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:22.806806 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce19500-b322-45e6-bf24-c4fc34a59a92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:22.807103 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce19500-b322-45e6-bf24-c4fc34a59a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:22.807370 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce19500-b322-45e6-bf24-c4fc34a59a92 admin admin] PATH_INFO: `/v3/projects/e7ff705387724e169c84fc07b1c5b498` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:22.808514 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce19500-b322-45e6-bf24-c4fc34a59a92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7ff705387724e169c84fc07b1c5b498)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:22.810455 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce19500-b322-45e6-bf24-c4fc34a59a92 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:22.816682 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:22.915755 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-7ce19500-b322-45e6-bf24-c4fc34a59a92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7ff705387724e169c84fc07b1c5b498', 'request_id': 'req-7ce19500-b322-45e6-bf24-c4fc34a59a92'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:22.921041 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-7ce19500-b322-45e6-bf24-c4fc34a59a92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7ff705387724e169c84fc07b1c5b498', 'request_id': 'req-7ce19500-b322-45e6-bf24-c4fc34a59a92'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:23.012314 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-7ce19500-b322-45e6-bf24-c4fc34a59a92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7ff705387724e169c84fc07b1c5b498', 'request_id': 'req-7ce19500-b322-45e6-bf24-c4fc34a59a92'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:23.026070 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-7ce19500-b322-45e6-bf24-c4fc34a59a92 admin admin] The token cache is being invalidate because project e7ff705387724e169c84fc07b1c5b498 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:51:23.026353 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-7ce19500-b322-45e6-bf24-c4fc34a59a92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:23.026999 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-7ce19500-b322-45e6-bf24-c4fc34a59a92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:23.028676 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1267/2557] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:51:22 2022] DELETE /identity/v3/projects/e7ff705387724e169c84fc07b1c5b498 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:51:23.032666 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7647c5-357f-469f-90ae-30fc73bf1338 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:23.034219 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-db7647c5-357f-469f-90ae-30fc73bf1338 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:23.047590 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7647c5-357f-469f-90ae-30fc73bf1338 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:23.105068 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a28bf3-c1a4-449b-963b-85fc7b8d3d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:23.105732 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a28bf3-c1a4-449b-963b-85fc7b8d3d5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:23.106014 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a28bf3-c1a4-449b-963b-85fc7b8d3d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:23.106290 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a28bf3-c1a4-449b-963b-85fc7b8d3d5a admin admin] PATH_INFO: `/v3/users/6fcfc35b845f46d69dc13fd41efcbcde` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:23.107426 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a28bf3-c1a4-449b-963b-85fc7b8d3d5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fcfc35b845f46d69dc13fd41efcbcde)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:23.109372 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a28bf3-c1a4-449b-963b-85fc7b8d3d5a admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:23.114803 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:23.443367 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-b8a28bf3-c1a4-449b-963b-85fc7b8d3d5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fcfc35b845f46d69dc13fd41efcbcde'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:23.444059 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-b8a28bf3-c1a4-449b-963b-85fc7b8d3d5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fcfc35b845f46d69dc13fd41efcbcde'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:23.444786 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-b8a28bf3-c1a4-449b-963b-85fc7b8d3d5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fcfc35b845f46d69dc13fd41efcbcde'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:23.543286 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-b8a28bf3-c1a4-449b-963b-85fc7b8d3d5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fcfc35b845f46d69dc13fd41efcbcde'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:23.557212 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-b8a28bf3-c1a4-449b-963b-85fc7b8d3d5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fcfc35b845f46d69dc13fd41efcbcde'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:23.569440 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1291/2558] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:51:23 2022] DELETE /identity/v3/users/6fcfc35b845f46d69dc13fd41efcbcde => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:51:23.830934 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce19500-b322-45e6-bf24-c4fc34a59a92 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:23.832369 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-7ce19500-b322-45e6-bf24-c4fc34a59a92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:23.848792 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce19500-b322-45e6-bf24-c4fc34a59a92 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:23.908523 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b062c0bc-93cd-4ff3-b7ce-783e3bdf93a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:23.909167 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b062c0bc-93cd-4ff3-b7ce-783e3bdf93a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:23.909462 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b062c0bc-93cd-4ff3-b7ce-783e3bdf93a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:23.909730 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b062c0bc-93cd-4ff3-b7ce-783e3bdf93a3 admin admin] PATH_INFO: `/v3/projects/a17c048213b44d3fa64d972bc082469c` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:23.910844 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b062c0bc-93cd-4ff3-b7ce-783e3bdf93a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a17c048213b44d3fa64d972bc082469c)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:23.912783 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b062c0bc-93cd-4ff3-b7ce-783e3bdf93a3 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:24.019181 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-b062c0bc-93cd-4ff3-b7ce-783e3bdf93a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a17c048213b44d3fa64d972bc082469c', 'request_id': 'req-b062c0bc-93cd-4ff3-b7ce-783e3bdf93a3'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:24.024883 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-b062c0bc-93cd-4ff3-b7ce-783e3bdf93a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a17c048213b44d3fa64d972bc082469c', 'request_id': 'req-b062c0bc-93cd-4ff3-b7ce-783e3bdf93a3'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:24.092263 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-b062c0bc-93cd-4ff3-b7ce-783e3bdf93a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a17c048213b44d3fa64d972bc082469c', 'request_id': 'req-b062c0bc-93cd-4ff3-b7ce-783e3bdf93a3'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:24.104580 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-b062c0bc-93cd-4ff3-b7ce-783e3bdf93a3 admin admin] The token cache is being invalidate because project a17c048213b44d3fa64d972bc082469c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:51:24.104871 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-b062c0bc-93cd-4ff3-b7ce-783e3bdf93a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:24.105546 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-b062c0bc-93cd-4ff3-b7ce-783e3bdf93a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:24.107039 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1268/2559] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:51:23 2022] DELETE /identity/v3/projects/a17c048213b44d3fa64d972bc082469c => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:51:24.112220 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef3c77d-dfb2-42ea-811a-351c1963256f None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:24.112725 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef3c77d-dfb2-42ea-811a-351c1963256f None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:24.112947 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef3c77d-dfb2-42ea-811a-351c1963256f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:24.135244 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-6ef3c77d-dfb2-42ea-811a-351c1963256f None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:51:24.136595 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-6ef3c77d-dfb2-42ea-811a-351c1963256f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:24.195918 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1292/2560] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:51:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:51:24.206621 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fac5d7-1a27-4eb0-b3e1-ffbcf06e7feb None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:24.206816 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fac5d7-1a27-4eb0-b3e1-ffbcf06e7feb None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:24.207026 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fac5d7-1a27-4eb0-b3e1-ffbcf06e7feb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:24.227717 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-e0fac5d7-1a27-4eb0-b3e1-ffbcf06e7feb None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:51:24.229238 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-e0fac5d7-1a27-4eb0-b3e1-ffbcf06e7feb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:24.237359 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1269/2561] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:51:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:51:24.241882 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef3c77d-dfb2-42ea-811a-351c1963256f None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:24.254969 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef3c77d-dfb2-42ea-811a-351c1963256f None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:24.260752 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bf0228-38eb-4bcc-8fcd-0534eb928e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:24.261412 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bf0228-38eb-4bcc-8fcd-0534eb928e26 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:24.261697 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bf0228-38eb-4bcc-8fcd-0534eb928e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:24.261968 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bf0228-38eb-4bcc-8fcd-0534eb928e26 admin admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:24.262633 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bf0228-38eb-4bcc-8fcd-0534eb928e26 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:51:24.263099 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bf0228-38eb-4bcc-8fcd-0534eb928e26 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:24.264775 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bf0228-38eb-4bcc-8fcd-0534eb928e26 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:24.285407 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1293/2562] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 24 06:51:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:51:24.285552 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fac5d7-1a27-4eb0-b3e1-ffbcf06e7feb None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:24.293284 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fac5d7-1a27-4eb0-b3e1-ffbcf06e7feb None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:24.298223 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e18e5a9-af76-43c0-9e01-e0c00d638bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:24.298827 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e18e5a9-af76-43c0-9e01-e0c00d638bc8 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:24.299115 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e18e5a9-af76-43c0-9e01-e0c00d638bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:24.299389 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e18e5a9-af76-43c0-9e01-e0c00d638bc8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:24.300069 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e18e5a9-af76-43c0-9e01-e0c00d638bc8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:24.303201 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e18e5a9-af76-43c0-9e01-e0c00d638bc8 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:24.320099 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:24.465798 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1270/2563] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:51:24 2022] POST /identity/v3/projects => generated 377 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:51:24.471346 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bf0228-38eb-4bcc-8fcd-0534eb928e26 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:24.481858 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bf0228-38eb-4bcc-8fcd-0534eb928e26 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:24.545131 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf3787c-6e03-4275-bb5c-4367a89bbe40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:24.545794 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf3787c-6e03-4275-bb5c-4367a89bbe40 admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:24.546079 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf3787c-6e03-4275-bb5c-4367a89bbe40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:24.546351 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf3787c-6e03-4275-bb5c-4367a89bbe40 admin admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:24.547002 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf3787c-6e03-4275-bb5c-4367a89bbe40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:24.548905 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf3787c-6e03-4275-bb5c-4367a89bbe40 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:24.670297 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1294/2564] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:51:24 2022] POST /identity/v3/users => generated 289 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:51:24.677656 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e18e5a9-af76-43c0-9e01-e0c00d638bc8 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:24.688134 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e18e5a9-af76-43c0-9e01-e0c00d638bc8 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:24.694450 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21dca3b-2f09-4adf-98eb-a1aa5918325a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:24.695075 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21dca3b-2f09-4adf-98eb-a1aa5918325a admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:24.695365 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21dca3b-2f09-4adf-98eb-a1aa5918325a admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:24.695641 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21dca3b-2f09-4adf-98eb-a1aa5918325a admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:24.696290 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21dca3b-2f09-4adf-98eb-a1aa5918325a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:24.698650 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21dca3b-2f09-4adf-98eb-a1aa5918325a admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:24.711378 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1271/2565] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:51:24 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:51:24.713464 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf3787c-6e03-4275-bb5c-4367a89bbe40 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:24.720027 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:24.726251 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf3787c-6e03-4275-bb5c-4367a89bbe40 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:24.731654 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238c9497-00ed-42f3-bdcb-8d0ae6adc643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:24.732301 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c9497-00ed-42f3-bdcb-8d0ae6adc643 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:24.732588 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c9497-00ed-42f3-bdcb-8d0ae6adc643 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:24.732867 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c9497-00ed-42f3-bdcb-8d0ae6adc643 admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:24.733550 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238c9497-00ed-42f3-bdcb-8d0ae6adc643 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:24.735126 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238c9497-00ed-42f3-bdcb-8d0ae6adc643 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:24.746411 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1295/2566] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:51:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:51:24.750883 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21dca3b-2f09-4adf-98eb-a1aa5918325a admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:24.761062 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21dca3b-2f09-4adf-98eb-a1aa5918325a admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:24.765772 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961232ca-fceb-40b6-a861-534b827fe5c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:24.766428 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961232ca-fceb-40b6-a861-534b827fe5c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:24.766706 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961232ca-fceb-40b6-a861-534b827fe5c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:24.766987 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961232ca-fceb-40b6-a861-534b827fe5c4 admin admin] PATH_INFO: `/v3/projects/9e1227b220ea446f934a2e8387d56a1b/users/f2e2d69d8cf044ffa7034391d8adb3d1/roles/e1f9b167c51c4a388cbc9594db8a465d` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:24.784156 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961232ca-fceb-40b6-a861-534b827fe5c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e1227b220ea446f934a2e8387d56a1b, user_id=f2e2d69d8cf044ffa7034391d8adb3d1, role_id=e1f9b167c51c4a388cbc9594db8a465d)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:24.786868 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961232ca-fceb-40b6-a861-534b827fe5c4 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:24.846880 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1272/2567] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:51:24 2022] PUT /identity/v3/projects/9e1227b220ea446f934a2e8387d56a1b/users/f2e2d69d8cf044ffa7034391d8adb3d1/roles/e1f9b167c51c4a388cbc9594db8a465d => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:51:24.857884 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6826bee-e035-42ba-95b5-fe1de408e4d8 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:24.858170 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6826bee-e035-42ba-95b5-fe1de408e4d8 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:24.858442 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6826bee-e035-42ba-95b5-fe1de408e4d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:24.888641 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-a6826bee-e035-42ba-95b5-fe1de408e4d8 None None] MFA Rules not processed for user `f2e2d69d8cf044ffa7034391d8adb3d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:51:24.889823 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-a6826bee-e035-42ba-95b5-fe1de408e4d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:24.955146 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1296/2568] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:51:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:51:24.967817 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961232ca-fceb-40b6-a861-534b827fe5c4 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:24.969682 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-961232ca-fceb-40b6-a861-534b827fe5c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:25.153974 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961232ca-fceb-40b6-a861-534b827fe5c4 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:25.157672 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dde5165-6f19-4e66-8678-160c0966c0ed service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b331b6ddfe46aeae7b52fcbbc27771', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:25.158369 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dde5165-6f19-4e66-8678-160c0966c0ed service glance] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:25.158637 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dde5165-6f19-4e66-8678-160c0966c0ed service glance] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:25.158908 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dde5165-6f19-4e66-8678-160c0966c0ed service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:25.167632 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dde5165-6f19-4e66-8678-160c0966c0ed service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:25.169101 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dde5165-6f19-4e66-8678-160c0966c0ed service glance] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:25.178892 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1273/2569] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:51:24 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:51:25.758062 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6826bee-e035-42ba-95b5-fe1de408e4d8 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:25.760435 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-a6826bee-e035-42ba-95b5-fe1de408e4d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:25.828751 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6826bee-e035-42ba-95b5-fe1de408e4d8 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:25.833629 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3ac0ff-abbe-43e2-9844-39643da8fe60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:25.834273 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3ac0ff-abbe-43e2-9844-39643da8fe60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:25.834556 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3ac0ff-abbe-43e2-9844-39643da8fe60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:25.834835 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3ac0ff-abbe-43e2-9844-39643da8fe60 admin admin] PATH_INFO: `/v3/users/81c51fc4d4c74fde8d5f73def83713e4` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:25.835999 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3ac0ff-abbe-43e2-9844-39643da8fe60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81c51fc4d4c74fde8d5f73def83713e4)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:25.838516 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3ac0ff-abbe-43e2-9844-39643da8fe60 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:26.063732 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-5f3ac0ff-abbe-43e2-9844-39643da8fe60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81c51fc4d4c74fde8d5f73def83713e4'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:26.064476 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-5f3ac0ff-abbe-43e2-9844-39643da8fe60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81c51fc4d4c74fde8d5f73def83713e4'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:26.065099 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-5f3ac0ff-abbe-43e2-9844-39643da8fe60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81c51fc4d4c74fde8d5f73def83713e4'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:26.136404 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-5f3ac0ff-abbe-43e2-9844-39643da8fe60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81c51fc4d4c74fde8d5f73def83713e4'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:26.151041 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-5f3ac0ff-abbe-43e2-9844-39643da8fe60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81c51fc4d4c74fde8d5f73def83713e4'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:26.159879 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:26.164268 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1297/2570] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:51:25 2022] DELETE /identity/v3/users/81c51fc4d4c74fde8d5f73def83713e4 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:51:26.168871 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc64cd20-868e-44be-9b27-6a7a46083e33 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:26.169570 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc64cd20-868e-44be-9b27-6a7a46083e33 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:26.169570 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc64cd20-868e-44be-9b27-6a7a46083e33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:26.188151 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-cc64cd20-868e-44be-9b27-6a7a46083e33 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:51:26.189378 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-cc64cd20-868e-44be-9b27-6a7a46083e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:26.256245 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1274/2571] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:51:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:51:26.267268 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3ac0ff-abbe-43e2-9844-39643da8fe60 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:26.268695 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-5f3ac0ff-abbe-43e2-9844-39643da8fe60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:26.332629 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3ac0ff-abbe-43e2-9844-39643da8fe60 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:26.336613 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5175ea-d24a-4537-ac3c-7d6d44ee62bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:26.337352 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5175ea-d24a-4537-ac3c-7d6d44ee62bd service neutron] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:26.337639 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5175ea-d24a-4537-ac3c-7d6d44ee62bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:26.337911 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5175ea-d24a-4537-ac3c-7d6d44ee62bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:26.349021 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5175ea-d24a-4537-ac3c-7d6d44ee62bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:26.350547 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5175ea-d24a-4537-ac3c-7d6d44ee62bd service neutron] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:26.360209 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1298/2572] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:51:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:51:26.833567 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc64cd20-868e-44be-9b27-6a7a46083e33 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:26.837233 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-cc64cd20-868e-44be-9b27-6a7a46083e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:26.848712 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc64cd20-868e-44be-9b27-6a7a46083e33 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:26.855771 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db31ff73-0e82-4b7b-b9ab-58b2fa9d68ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:26.856810 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db31ff73-0e82-4b7b-b9ab-58b2fa9d68ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:26.862603 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db31ff73-0e82-4b7b-b9ab-58b2fa9d68ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:26.862603 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db31ff73-0e82-4b7b-b9ab-58b2fa9d68ef admin admin] PATH_INFO: `/v3/projects/89bc91d11de14892b9598d5ced6f3848` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:26.862603 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db31ff73-0e82-4b7b-b9ab-58b2fa9d68ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=89bc91d11de14892b9598d5ced6f3848)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:26.862603 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db31ff73-0e82-4b7b-b9ab-58b2fa9d68ef admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:26.992545 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-db31ff73-0e82-4b7b-b9ab-58b2fa9d68ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89bc91d11de14892b9598d5ced6f3848', 'request_id': 'req-db31ff73-0e82-4b7b-b9ab-58b2fa9d68ef'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:26.994460 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-db31ff73-0e82-4b7b-b9ab-58b2fa9d68ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89bc91d11de14892b9598d5ced6f3848', 'request_id': 'req-db31ff73-0e82-4b7b-b9ab-58b2fa9d68ef'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:26.996493 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:27.154767 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-db31ff73-0e82-4b7b-b9ab-58b2fa9d68ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89bc91d11de14892b9598d5ced6f3848', 'request_id': 'req-db31ff73-0e82-4b7b-b9ab-58b2fa9d68ef'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:27.169055 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-db31ff73-0e82-4b7b-b9ab-58b2fa9d68ef admin admin] The token cache is being invalidate because project 89bc91d11de14892b9598d5ced6f3848 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:51:27.169767 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-db31ff73-0e82-4b7b-b9ab-58b2fa9d68ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:27.170068 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-db31ff73-0e82-4b7b-b9ab-58b2fa9d68ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:27.171722 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1275/2573] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:51:26 2022] DELETE /identity/v3/projects/89bc91d11de14892b9598d5ced6f3848 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:51:27.177279 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc9938-d32d-48ad-8caa-f39ed8f289e6 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:27.177786 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc9938-d32d-48ad-8caa-f39ed8f289e6 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:27.177786 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc9938-d32d-48ad-8caa-f39ed8f289e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:27.206434 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-8bfc9938-d32d-48ad-8caa-f39ed8f289e6 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:51:27.206434 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-8bfc9938-d32d-48ad-8caa-f39ed8f289e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:27.265962 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1299/2574] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:51:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:51:27.274959 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566fdd2b-788c-440c-bbff-417ec42ca88a None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:27.275232 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566fdd2b-788c-440c-bbff-417ec42ca88a None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:27.275647 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566fdd2b-788c-440c-bbff-417ec42ca88a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:27.295961 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-566fdd2b-788c-440c-bbff-417ec42ca88a None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:51:27.297242 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-566fdd2b-788c-440c-bbff-417ec42ca88a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:27.305544 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1276/2575] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:51:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:51:27.310368 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfc9938-d32d-48ad-8caa-f39ed8f289e6 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:27.320741 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfc9938-d32d-48ad-8caa-f39ed8f289e6 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:27.326033 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6919d5e-4740-4a5e-8049-936d6a3f29da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:27.326643 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6919d5e-4740-4a5e-8049-936d6a3f29da admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:27.326926 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6919d5e-4740-4a5e-8049-936d6a3f29da admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:27.327201 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6919d5e-4740-4a5e-8049-936d6a3f29da admin admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:27.327852 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6919d5e-4740-4a5e-8049-936d6a3f29da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:51:27.328152 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6919d5e-4740-4a5e-8049-936d6a3f29da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:27.329906 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6919d5e-4740-4a5e-8049-936d6a3f29da admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:27.343763 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1300/2576] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 24 06:51:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:51:27.348389 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566fdd2b-788c-440c-bbff-417ec42ca88a None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:27.358858 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566fdd2b-788c-440c-bbff-417ec42ca88a None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:27.363530 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2235a59-f70a-407f-bcff-bdf1e949f7d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:27.364129 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2235a59-f70a-407f-bcff-bdf1e949f7d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:27.364408 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2235a59-f70a-407f-bcff-bdf1e949f7d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:27.364686 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2235a59-f70a-407f-bcff-bdf1e949f7d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:27.365365 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2235a59-f70a-407f-bcff-bdf1e949f7d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:27.367256 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2235a59-f70a-407f-bcff-bdf1e949f7d7 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:27.463106 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1277/2577] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:51:27 2022] POST /identity/v3/projects => generated 365 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:51:27.467689 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6919d5e-4740-4a5e-8049-936d6a3f29da admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:27.478197 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6919d5e-4740-4a5e-8049-936d6a3f29da admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:27.548257 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a37f69b-91f8-424a-b9a1-82560e8d03b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:27.549392 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a37f69b-91f8-424a-b9a1-82560e8d03b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:27.549673 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a37f69b-91f8-424a-b9a1-82560e8d03b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:27.549945 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a37f69b-91f8-424a-b9a1-82560e8d03b2 admin admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:27.550578 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a37f69b-91f8-424a-b9a1-82560e8d03b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:27.552654 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a37f69b-91f8-424a-b9a1-82560e8d03b2 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:27.651789 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1301/2578] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:51:27 2022] POST /identity/v3/users => generated 283 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:51:27.656023 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2235a59-f70a-407f-bcff-bdf1e949f7d7 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:27.667177 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2235a59-f70a-407f-bcff-bdf1e949f7d7 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:27.671796 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7daacd-b316-4224-951f-b136c737114d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:27.672395 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7daacd-b316-4224-951f-b136c737114d admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:27.672673 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7daacd-b316-4224-951f-b136c737114d admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:27.672945 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7daacd-b316-4224-951f-b136c737114d admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:27.673620 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7daacd-b316-4224-951f-b136c737114d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:27.675201 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7daacd-b316-4224-951f-b136c737114d admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:27.686172 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1278/2579] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:51:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:51:27.690381 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a37f69b-91f8-424a-b9a1-82560e8d03b2 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:27.701774 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a37f69b-91f8-424a-b9a1-82560e8d03b2 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:27.706798 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ac136a-a1dd-4bb6-b1cb-11afded69621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:27.707460 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ac136a-a1dd-4bb6-b1cb-11afded69621 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:27.707752 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ac136a-a1dd-4bb6-b1cb-11afded69621 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:27.708032 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ac136a-a1dd-4bb6-b1cb-11afded69621 admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:27.708680 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ac136a-a1dd-4bb6-b1cb-11afded69621 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:27.710319 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ac136a-a1dd-4bb6-b1cb-11afded69621 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:27.723742 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1302/2580] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:51:27 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:51:27.728061 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7daacd-b316-4224-951f-b136c737114d admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:27.738234 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7daacd-b316-4224-951f-b136c737114d admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:27.744367 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479fd027-bacc-4625-8ce3-dbf945056293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:27.745056 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479fd027-bacc-4625-8ce3-dbf945056293 admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:27.745375 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479fd027-bacc-4625-8ce3-dbf945056293 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:27.745770 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479fd027-bacc-4625-8ce3-dbf945056293 admin admin] PATH_INFO: `/v3/projects/6cc9120b955f4804985fb61b6a22ab87/users/8cbadf33fe01430ca66dcd409ee8eda1/roles/e1f9b167c51c4a388cbc9594db8a465d` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:27.768519 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479fd027-bacc-4625-8ce3-dbf945056293 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cc9120b955f4804985fb61b6a22ab87, user_id=8cbadf33fe01430ca66dcd409ee8eda1, role_id=e1f9b167c51c4a388cbc9594db8a465d)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:27.771214 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479fd027-bacc-4625-8ce3-dbf945056293 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:27.870554 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1279/2581] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:51:27 2022] PUT /identity/v3/projects/6cc9120b955f4804985fb61b6a22ab87/users/8cbadf33fe01430ca66dcd409ee8eda1/roles/e1f9b167c51c4a388cbc9594db8a465d => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:51:27.881509 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8616c5-48ef-4437-8698-490389c00095 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:27.881855 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8616c5-48ef-4437-8698-490389c00095 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:27.882066 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8616c5-48ef-4437-8698-490389c00095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:27.902708 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:27.912215 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-7a8616c5-48ef-4437-8698-490389c00095 None None] MFA Rules not processed for user `8cbadf33fe01430ca66dcd409ee8eda1`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:51:27.913513 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-7a8616c5-48ef-4437-8698-490389c00095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:27.984350 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1303/2582] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:51:27 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:51:27.996643 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479fd027-bacc-4625-8ce3-dbf945056293 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:27.999027 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-479fd027-bacc-4625-8ce3-dbf945056293 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:28.065215 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479fd027-bacc-4625-8ce3-dbf945056293 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:28.068943 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a1d062-d1df-436d-9dd1-18683bc5800c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36639dce4f97412f89d71367f650b901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:28.069584 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a1d062-d1df-436d-9dd1-18683bc5800c service cinder] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:28.069865 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a1d062-d1df-436d-9dd1-18683bc5800c service cinder] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:28.070152 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a1d062-d1df-436d-9dd1-18683bc5800c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:28.080218 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a1d062-d1df-436d-9dd1-18683bc5800c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:28.081688 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a1d062-d1df-436d-9dd1-18683bc5800c service cinder] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:28.092318 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1280/2583] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:51:27 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:51:37.986700 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=28 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:38.086640 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:INSERT=3,keystone:DELETE=1,keystone:SELECT=57 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:45.131705 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8616c5-48ef-4437-8698-490389c00095 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:45.141342 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8616c5-48ef-4437-8698-490389c00095 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:45.145769 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b131775b-8ce6-477e-b170-1691bc87513f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36639dce4f97412f89d71367f650b901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:45.146363 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b131775b-8ce6-477e-b170-1691bc87513f service cinder] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:45.146638 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b131775b-8ce6-477e-b170-1691bc87513f service cinder] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:45.146903 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b131775b-8ce6-477e-b170-1691bc87513f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:45.148648 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-b131775b-8ce6-477e-b170-1691bc87513f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:45.231412 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b131775b-8ce6-477e-b170-1691bc87513f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:45.232862 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b131775b-8ce6-477e-b170-1691bc87513f service cinder] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:45.250768 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1304/2584] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:51:45 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:51:55.248227 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=41 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:51:56.350635 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a1d062-d1df-436d-9dd1-18683bc5800c service cinder] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:56.352126 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-f0a1d062-d1df-436d-9dd1-18683bc5800c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:56.417718 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a1d062-d1df-436d-9dd1-18683bc5800c service cinder] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:56.423109 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f89ee7d-05c3-4f9a-ae4d-84eb8bea86dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:56.423781 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f89ee7d-05c3-4f9a-ae4d-84eb8bea86dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:56.424065 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f89ee7d-05c3-4f9a-ae4d-84eb8bea86dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:56.424342 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f89ee7d-05c3-4f9a-ae4d-84eb8bea86dd admin admin] PATH_INFO: `/v3/users/f2e2d69d8cf044ffa7034391d8adb3d1` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:56.425721 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f89ee7d-05c3-4f9a-ae4d-84eb8bea86dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2e2d69d8cf044ffa7034391d8adb3d1)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:56.427791 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f89ee7d-05c3-4f9a-ae4d-84eb8bea86dd admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:56.637536 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-3f89ee7d-05c3-4f9a-ae4d-84eb8bea86dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2e2d69d8cf044ffa7034391d8adb3d1'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:56.638300 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-3f89ee7d-05c3-4f9a-ae4d-84eb8bea86dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2e2d69d8cf044ffa7034391d8adb3d1'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:56.639032 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-3f89ee7d-05c3-4f9a-ae4d-84eb8bea86dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2e2d69d8cf044ffa7034391d8adb3d1'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:56.647132 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-3f89ee7d-05c3-4f9a-ae4d-84eb8bea86dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2e2d69d8cf044ffa7034391d8adb3d1'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:56.722671 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-3f89ee7d-05c3-4f9a-ae4d-84eb8bea86dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2e2d69d8cf044ffa7034391d8adb3d1'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:56.744470 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1281/2585] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:51:56 2022] DELETE /identity/v3/users/f2e2d69d8cf044ffa7034391d8adb3d1 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:51:56.749099 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c60d9-5fbd-4762-ab8b-f620741a632e None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:56.749785 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c60d9-5fbd-4762-ab8b-f620741a632e None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:56.750076 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c60d9-5fbd-4762-ab8b-f620741a632e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:56.770981 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-231c60d9-5fbd-4762-ab8b-f620741a632e None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:51:56.773076 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-231c60d9-5fbd-4762-ab8b-f620741a632e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:56.845433 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1305/2586] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:51:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:51:56.856625 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f89ee7d-05c3-4f9a-ae4d-84eb8bea86dd admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:56.858120 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-3f89ee7d-05c3-4f9a-ae4d-84eb8bea86dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:56.919716 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f89ee7d-05c3-4f9a-ae4d-84eb8bea86dd admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:56.923288 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda453d8-aa8e-4eaf-8cf4-cef93cd92b73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:56.923893 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda453d8-aa8e-4eaf-8cf4-cef93cd92b73 service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:56.924174 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda453d8-aa8e-4eaf-8cf4-cef93cd92b73 service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:56.924485 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda453d8-aa8e-4eaf-8cf4-cef93cd92b73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:56.934189 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda453d8-aa8e-4eaf-8cf4-cef93cd92b73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:56.935605 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda453d8-aa8e-4eaf-8cf4-cef93cd92b73 service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:56.946141 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1282/2587] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:51:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:51:57.391364 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231c60d9-5fbd-4762-ab8b-f620741a632e None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:51:57.392823 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-231c60d9-5fbd-4762-ab8b-f620741a632e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:51:57.407077 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231c60d9-5fbd-4762-ab8b-f620741a632e None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:51:57.412267 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9be3eb6-ff63-4d3c-96a9-74b3e7b5a625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:51:57.413671 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9be3eb6-ff63-4d3c-96a9-74b3e7b5a625 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:51:57.413953 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9be3eb6-ff63-4d3c-96a9-74b3e7b5a625 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:51:57.414219 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9be3eb6-ff63-4d3c-96a9-74b3e7b5a625 admin admin] PATH_INFO: `/v3/projects/9e1227b220ea446f934a2e8387d56a1b` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:51:57.415461 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9be3eb6-ff63-4d3c-96a9-74b3e7b5a625 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e1227b220ea446f934a2e8387d56a1b)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:51:57.417546 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9be3eb6-ff63-4d3c-96a9-74b3e7b5a625 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:51:57.518002 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-a9be3eb6-ff63-4d3c-96a9-74b3e7b5a625 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e1227b220ea446f934a2e8387d56a1b', 'request_id': 'req-a9be3eb6-ff63-4d3c-96a9-74b3e7b5a625'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:57.588585 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-a9be3eb6-ff63-4d3c-96a9-74b3e7b5a625 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e1227b220ea446f934a2e8387d56a1b', 'request_id': 'req-a9be3eb6-ff63-4d3c-96a9-74b3e7b5a625'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:57.596453 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-a9be3eb6-ff63-4d3c-96a9-74b3e7b5a625 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e1227b220ea446f934a2e8387d56a1b', 'request_id': 'req-a9be3eb6-ff63-4d3c-96a9-74b3e7b5a625'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:57.607588 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-a9be3eb6-ff63-4d3c-96a9-74b3e7b5a625 admin admin] The token cache is being invalidate because project 9e1227b220ea446f934a2e8387d56a1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81723) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:51:57.608044 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-a9be3eb6-ff63-4d3c-96a9-74b3e7b5a625 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:57.608699 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-a9be3eb6-ff63-4d3c-96a9-74b3e7b5a625 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:51:57.610319 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1306/2588] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:51:57 2022] DELETE /identity/v3/projects/9e1227b220ea446f934a2e8387d56a1b => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:52:06.691880 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda453d8-aa8e-4eaf-8cf4-cef93cd92b73 service neutron] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:06.693335 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-bda453d8-aa8e-4eaf-8cf4-cef93cd92b73 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:06.737415 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:52:06.827900 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda453d8-aa8e-4eaf-8cf4-cef93cd92b73 service neutron] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:06.922783 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c876ed02-657e-459f-bc50-d56c33a7ff60 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4f49c1b08c460f8ba6c451c9c92c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:06.922783 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c876ed02-657e-459f-bc50-d56c33a7ff60 service nova] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:06.922783 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c876ed02-657e-459f-bc50-d56c33a7ff60 service nova] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:06.922783 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c876ed02-657e-459f-bc50-d56c33a7ff60 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:06.923263 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-c876ed02-657e-459f-bc50-d56c33a7ff60 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:06.937393 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c876ed02-657e-459f-bc50-d56c33a7ff60 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:06.939264 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c876ed02-657e-459f-bc50-d56c33a7ff60 service nova] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:06.952384 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1283/2589] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:52:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:52:07.604453 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:52:15.112368 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9be3eb6-ff63-4d3c-96a9-74b3e7b5a625 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:15.113936 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-a9be3eb6-ff63-4d3c-96a9-74b3e7b5a625 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:15.174047 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9be3eb6-ff63-4d3c-96a9-74b3e7b5a625 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:15.180409 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51de904-9a57-4687-b512-a6885e2ddf92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:15.180409 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51de904-9a57-4687-b512-a6885e2ddf92 service neutron] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:15.180409 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51de904-9a57-4687-b512-a6885e2ddf92 service neutron] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:15.180409 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51de904-9a57-4687-b512-a6885e2ddf92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:15.182248 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-f51de904-9a57-4687-b512-a6885e2ddf92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:15.249429 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51de904-9a57-4687-b512-a6885e2ddf92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:15.250864 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51de904-9a57-4687-b512-a6885e2ddf92 service neutron] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:15.260423 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1307/2590] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:52:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:52:17.146425 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=52 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:52:19.058940 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c876ed02-657e-459f-bc50-d56c33a7ff60 service nova] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:19.060499 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-c876ed02-657e-459f-bc50-d56c33a7ff60 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:19.074966 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c876ed02-657e-459f-bc50-d56c33a7ff60 service nova] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:19.080638 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a16270-5b06-4c23-9229-8982665c32e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:19.081299 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a16270-5b06-4c23-9229-8982665c32e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:19.081581 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a16270-5b06-4c23-9229-8982665c32e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:19.082211 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a16270-5b06-4c23-9229-8982665c32e0 admin admin] PATH_INFO: `/v3/users/02ad6cdc4ffe4917a7bbf6814297f980` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:19.083152 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a16270-5b06-4c23-9229-8982665c32e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=02ad6cdc4ffe4917a7bbf6814297f980)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:19.085056 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a16270-5b06-4c23-9229-8982665c32e0 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:19.345847 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-94a16270-5b06-4c23-9229-8982665c32e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02ad6cdc4ffe4917a7bbf6814297f980'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:19.346435 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-94a16270-5b06-4c23-9229-8982665c32e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02ad6cdc4ffe4917a7bbf6814297f980'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:19.347032 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-94a16270-5b06-4c23-9229-8982665c32e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02ad6cdc4ffe4917a7bbf6814297f980'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:19.354042 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-94a16270-5b06-4c23-9229-8982665c32e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02ad6cdc4ffe4917a7bbf6814297f980'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:19.445286 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-94a16270-5b06-4c23-9229-8982665c32e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02ad6cdc4ffe4917a7bbf6814297f980'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:19.465435 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1284/2591] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:52:19 2022] DELETE /identity/v3/users/02ad6cdc4ffe4917a7bbf6814297f980 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:52:19.733350 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51de904-9a57-4687-b512-a6885e2ddf92 service neutron] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:19.746710 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51de904-9a57-4687-b512-a6885e2ddf92 service neutron] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:19.808180 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79dcc45-a347-4eb9-b6ee-4bab4d9000be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:19.808813 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79dcc45-a347-4eb9-b6ee-4bab4d9000be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:19.809096 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79dcc45-a347-4eb9-b6ee-4bab4d9000be admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:19.809402 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79dcc45-a347-4eb9-b6ee-4bab4d9000be admin admin] PATH_INFO: `/v3/projects/b91004b0497d487ea8f1f454c03bbca0` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:19.810869 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79dcc45-a347-4eb9-b6ee-4bab4d9000be admin admin] RBAC: Authorizing `identity:delete_project(project_id=b91004b0497d487ea8f1f454c03bbca0)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:19.812811 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79dcc45-a347-4eb9-b6ee-4bab4d9000be admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:19.927810 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-b79dcc45-a347-4eb9-b6ee-4bab4d9000be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b91004b0497d487ea8f1f454c03bbca0', 'request_id': 'req-b79dcc45-a347-4eb9-b6ee-4bab4d9000be'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:19.930393 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:52:20.018642 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-b79dcc45-a347-4eb9-b6ee-4bab4d9000be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b91004b0497d487ea8f1f454c03bbca0', 'request_id': 'req-b79dcc45-a347-4eb9-b6ee-4bab4d9000be'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:20.025712 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-b79dcc45-a347-4eb9-b6ee-4bab4d9000be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b91004b0497d487ea8f1f454c03bbca0', 'request_id': 'req-b79dcc45-a347-4eb9-b6ee-4bab4d9000be'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:20.037880 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-b79dcc45-a347-4eb9-b6ee-4bab4d9000be admin admin] The token cache is being invalidate because project b91004b0497d487ea8f1f454c03bbca0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81723) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:52:20.038163 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-b79dcc45-a347-4eb9-b6ee-4bab4d9000be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:20.038818 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-b79dcc45-a347-4eb9-b6ee-4bab4d9000be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:20.040393 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1308/2592] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:52:19 2022] DELETE /identity/v3/projects/b91004b0497d487ea8f1f454c03bbca0 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:52:20.045716 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a172164e-ffc4-4267-b832-38c91ff14a1b None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:20.048253 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a172164e-ffc4-4267-b832-38c91ff14a1b None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:20.048253 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a172164e-ffc4-4267-b832-38c91ff14a1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:20.067097 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-a172164e-ffc4-4267-b832-38c91ff14a1b None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:52:20.068317 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-a172164e-ffc4-4267-b832-38c91ff14a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:20.129242 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1285/2593] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:52:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:52:20.140732 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83df21e2-77cd-4f69-8087-71bdfa0979ce None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:20.141014 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83df21e2-77cd-4f69-8087-71bdfa0979ce None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:20.141355 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83df21e2-77cd-4f69-8087-71bdfa0979ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:20.162583 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-83df21e2-77cd-4f69-8087-71bdfa0979ce None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:52:20.163784 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-83df21e2-77cd-4f69-8087-71bdfa0979ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:20.171731 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1309/2594] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:52:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:52:20.176002 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a172164e-ffc4-4267-b832-38c91ff14a1b None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:20.186111 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a172164e-ffc4-4267-b832-38c91ff14a1b None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:20.190685 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3c2d07-fde5-4f55-a2f2-71afacf8d867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:20.191297 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3c2d07-fde5-4f55-a2f2-71afacf8d867 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:20.191576 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3c2d07-fde5-4f55-a2f2-71afacf8d867 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:20.191847 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3c2d07-fde5-4f55-a2f2-71afacf8d867 admin admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:20.192496 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3c2d07-fde5-4f55-a2f2-71afacf8d867 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:52:20.192799 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3c2d07-fde5-4f55-a2f2-71afacf8d867 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:20.194417 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3c2d07-fde5-4f55-a2f2-71afacf8d867 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:20.207128 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1286/2595] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 24 06:52:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:52:20.211429 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83df21e2-77cd-4f69-8087-71bdfa0979ce None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:20.222011 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83df21e2-77cd-4f69-8087-71bdfa0979ce None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:20.227014 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad732a2-86ea-464e-8e5c-3e37c3b7eee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:20.227630 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad732a2-86ea-464e-8e5c-3e37c3b7eee5 admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:20.227908 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad732a2-86ea-464e-8e5c-3e37c3b7eee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:20.228174 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad732a2-86ea-464e-8e5c-3e37c3b7eee5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:20.228818 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad732a2-86ea-464e-8e5c-3e37c3b7eee5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:20.230726 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad732a2-86ea-464e-8e5c-3e37c3b7eee5 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:20.330689 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1310/2596] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:52:20 2022] POST /identity/v3/projects => generated 365 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:52:20.335340 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3c2d07-fde5-4f55-a2f2-71afacf8d867 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:20.346766 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3c2d07-fde5-4f55-a2f2-71afacf8d867 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:20.403050 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8017e666-c166-408d-9699-d69993dbd1ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:20.403662 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8017e666-c166-408d-9699-d69993dbd1ad admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:20.403945 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8017e666-c166-408d-9699-d69993dbd1ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:20.404239 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8017e666-c166-408d-9699-d69993dbd1ad admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:20.405471 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8017e666-c166-408d-9699-d69993dbd1ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:20.407366 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8017e666-c166-408d-9699-d69993dbd1ad admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:20.415298 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:52:20.511078 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1287/2597] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:52:20 2022] POST /identity/v3/users => generated 283 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:52:20.516638 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad732a2-86ea-464e-8e5c-3e37c3b7eee5 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:20.527181 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad732a2-86ea-464e-8e5c-3e37c3b7eee5 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:20.532663 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adffc43a-ac08-4c81-889f-8851a19ac5b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:20.533293 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adffc43a-ac08-4c81-889f-8851a19ac5b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:20.533580 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adffc43a-ac08-4c81-889f-8851a19ac5b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:20.533850 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adffc43a-ac08-4c81-889f-8851a19ac5b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:20.534483 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adffc43a-ac08-4c81-889f-8851a19ac5b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:20.536058 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adffc43a-ac08-4c81-889f-8851a19ac5b1 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:20.547383 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1311/2598] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:52:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:52:20.551556 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8017e666-c166-408d-9699-d69993dbd1ad admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:20.562183 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8017e666-c166-408d-9699-d69993dbd1ad admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:20.570944 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65bc22a-be36-4b95-9afe-0daddca7e947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:20.570944 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65bc22a-be36-4b95-9afe-0daddca7e947 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:20.570944 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65bc22a-be36-4b95-9afe-0daddca7e947 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:20.570944 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65bc22a-be36-4b95-9afe-0daddca7e947 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:20.571318 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65bc22a-be36-4b95-9afe-0daddca7e947 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:20.571318 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65bc22a-be36-4b95-9afe-0daddca7e947 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:20.581727 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1288/2599] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:52:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:52:20.586083 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adffc43a-ac08-4c81-889f-8851a19ac5b1 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:20.597331 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adffc43a-ac08-4c81-889f-8851a19ac5b1 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:20.604728 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd05b00-36f2-4644-9656-c1735ed0a2fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:20.605904 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd05b00-36f2-4644-9656-c1735ed0a2fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:20.606419 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd05b00-36f2-4644-9656-c1735ed0a2fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:20.606917 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd05b00-36f2-4644-9656-c1735ed0a2fd admin admin] PATH_INFO: `/v3/projects/0d92e2cbdd234a91b37f51cf7b31e29d/users/0d31b23b1fa648a19110aa8bcddad1df/roles/e1f9b167c51c4a388cbc9594db8a465d` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:20.624643 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd05b00-36f2-4644-9656-c1735ed0a2fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d92e2cbdd234a91b37f51cf7b31e29d, user_id=0d31b23b1fa648a19110aa8bcddad1df, role_id=e1f9b167c51c4a388cbc9594db8a465d)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:20.633412 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd05b00-36f2-4644-9656-c1735ed0a2fd admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:20.732997 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1312/2600] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:52:20 2022] PUT /identity/v3/projects/0d92e2cbdd234a91b37f51cf7b31e29d/users/0d31b23b1fa648a19110aa8bcddad1df/roles/e1f9b167c51c4a388cbc9594db8a465d => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:52:20.743211 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4590fae6-5927-41d7-bdee-f236e4a974af None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:20.753040 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4590fae6-5927-41d7-bdee-f236e4a974af None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:20.753040 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4590fae6-5927-41d7-bdee-f236e4a974af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:20.773888 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-4590fae6-5927-41d7-bdee-f236e4a974af None None] MFA Rules not processed for user `0d31b23b1fa648a19110aa8bcddad1df`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:52:20.775203 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-4590fae6-5927-41d7-bdee-f236e4a974af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:20.841245 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1289/2601] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:52:20 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:52:20.846068 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd05b00-36f2-4644-9656-c1735ed0a2fd admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:20.908263 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd05b00-36f2-4644-9656-c1735ed0a2fd admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:20.913277 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6fb2cb-9289-4964-a6ca-091c2be42bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:20.913884 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6fb2cb-9289-4964-a6ca-091c2be42bdd admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:20.914175 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6fb2cb-9289-4964-a6ca-091c2be42bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:20.914460 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6fb2cb-9289-4964-a6ca-091c2be42bdd admin admin] PATH_INFO: `/v3/projects` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:20.915080 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6fb2cb-9289-4964-a6ca-091c2be42bdd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:20.917067 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6fb2cb-9289-4964-a6ca-091c2be42bdd admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:21.037918 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1313/2602] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:52:20 2022] POST /identity/v3/projects => generated 365 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:52:21.041989 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4590fae6-5927-41d7-bdee-f236e4a974af None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:21.053481 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4590fae6-5927-41d7-bdee-f236e4a974af None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:21.122688 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a8815f-7f7c-4e4f-a8b7-671f65f50a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:21.124090 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a8815f-7f7c-4e4f-a8b7-671f65f50a8a admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:21.124503 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a8815f-7f7c-4e4f-a8b7-671f65f50a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:21.124862 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a8815f-7f7c-4e4f-a8b7-671f65f50a8a admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:21.125536 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a8815f-7f7c-4e4f-a8b7-671f65f50a8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:21.127449 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a8815f-7f7c-4e4f-a8b7-671f65f50a8a admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:21.270356 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1290/2603] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:52:21 2022] POST /identity/v3/users => generated 289 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:52:21.274395 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6fb2cb-9289-4964-a6ca-091c2be42bdd admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:21.286186 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6fb2cb-9289-4964-a6ca-091c2be42bdd admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:21.290775 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b799825e-288a-42e0-a1f3-0b4e7e202c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:21.291324 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b799825e-288a-42e0-a1f3-0b4e7e202c86 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:21.291602 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b799825e-288a-42e0-a1f3-0b4e7e202c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:21.291873 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b799825e-288a-42e0-a1f3-0b4e7e202c86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:21.292536 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b799825e-288a-42e0-a1f3-0b4e7e202c86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:21.294162 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b799825e-288a-42e0-a1f3-0b4e7e202c86 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:21.305737 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1314/2604] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:52:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:52:21.309951 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a8815f-7f7c-4e4f-a8b7-671f65f50a8a admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:21.319985 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a8815f-7f7c-4e4f-a8b7-671f65f50a8a admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:21.324546 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fc58e0-1964-4582-9e71-e4c22a13b161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:21.325257 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fc58e0-1964-4582-9e71-e4c22a13b161 admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:21.325542 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fc58e0-1964-4582-9e71-e4c22a13b161 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:21.325817 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fc58e0-1964-4582-9e71-e4c22a13b161 admin admin] PATH_INFO: `/v3/projects/300eab25dfd747dc87bd60d95b21f4e8/users/2a7537860cae40f787818766fb5d18ce/roles/486144ad1bcd417e86acdbc1d14327bf` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:21.340795 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fc58e0-1964-4582-9e71-e4c22a13b161 admin admin] RBAC: Authorizing `identity:create_grant(project_id=300eab25dfd747dc87bd60d95b21f4e8, user_id=2a7537860cae40f787818766fb5d18ce, role_id=486144ad1bcd417e86acdbc1d14327bf)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:21.344161 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fc58e0-1964-4582-9e71-e4c22a13b161 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:21.413614 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1291/2605] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:52:21 2022] PUT /identity/v3/projects/300eab25dfd747dc87bd60d95b21f4e8/users/2a7537860cae40f787818766fb5d18ce/roles/486144ad1bcd417e86acdbc1d14327bf => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:52:21.417989 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b799825e-288a-42e0-a1f3-0b4e7e202c86 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:21.429158 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b799825e-288a-42e0-a1f3-0b4e7e202c86 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:21.476509 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:52:21.488290 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3f6d5c-b25b-42e2-911d-875ed58cab72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:21.488980 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f6d5c-b25b-42e2-911d-875ed58cab72 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:21.489337 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f6d5c-b25b-42e2-911d-875ed58cab72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:21.489671 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3f6d5c-b25b-42e2-911d-875ed58cab72 admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:21.490336 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3f6d5c-b25b-42e2-911d-875ed58cab72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:21.492016 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3f6d5c-b25b-42e2-911d-875ed58cab72 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:21.502953 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1315/2606] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:52:21 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:52:21.507339 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fc58e0-1964-4582-9e71-e4c22a13b161 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:21.518159 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fc58e0-1964-4582-9e71-e4c22a13b161 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:21.523328 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37db6c5-19aa-4439-b3dc-888c9ef89685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:21.523997 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37db6c5-19aa-4439-b3dc-888c9ef89685 admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:21.524277 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37db6c5-19aa-4439-b3dc-888c9ef89685 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:21.524560 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37db6c5-19aa-4439-b3dc-888c9ef89685 admin admin] PATH_INFO: `/v3/projects/300eab25dfd747dc87bd60d95b21f4e8/users/2a7537860cae40f787818766fb5d18ce/roles/486144ad1bcd417e86acdbc1d14327bf` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:21.527474 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37db6c5-19aa-4439-b3dc-888c9ef89685 admin admin] RBAC: Authorizing `identity:create_grant(project_id=300eab25dfd747dc87bd60d95b21f4e8, user_id=2a7537860cae40f787818766fb5d18ce, role_id=486144ad1bcd417e86acdbc1d14327bf)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:21.531557 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37db6c5-19aa-4439-b3dc-888c9ef89685 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:21.542411 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1292/2607] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:52:21 2022] PUT /identity/v3/projects/300eab25dfd747dc87bd60d95b21f4e8/users/2a7537860cae40f787818766fb5d18ce/roles/486144ad1bcd417e86acdbc1d14327bf => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:52:21.559272 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908e0231-c1cd-423c-b700-081cf9d9b7cf None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:21.559914 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908e0231-c1cd-423c-b700-081cf9d9b7cf None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:21.560361 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908e0231-c1cd-423c-b700-081cf9d9b7cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:21.603559 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-908e0231-c1cd-423c-b700-081cf9d9b7cf None None] MFA Rules not processed for user `2a7537860cae40f787818766fb5d18ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:52:21.605267 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-908e0231-c1cd-423c-b700-081cf9d9b7cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:21.697017 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1316/2608] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:52:21 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:52:21.709016 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37db6c5-19aa-4439-b3dc-888c9ef89685 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:21.710828 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-f37db6c5-19aa-4439-b3dc-888c9ef89685 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:21.735824 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:52:21.771809 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37db6c5-19aa-4439-b3dc-888c9ef89685 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:21.776098 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60147fcf-9bc7-4cca-8ef3-2fb30a8be441 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36639dce4f97412f89d71367f650b901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:21.776735 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60147fcf-9bc7-4cca-8ef3-2fb30a8be441 service cinder] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:21.777020 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60147fcf-9bc7-4cca-8ef3-2fb30a8be441 service cinder] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:21.777330 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60147fcf-9bc7-4cca-8ef3-2fb30a8be441 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:21.787531 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60147fcf-9bc7-4cca-8ef3-2fb30a8be441 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:21.788976 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60147fcf-9bc7-4cca-8ef3-2fb30a8be441 service cinder] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:21.799908 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1293/2609] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:52:21 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:52:22.681558 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908e0231-c1cd-423c-b700-081cf9d9b7cf None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:22.749091 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908e0231-c1cd-423c-b700-081cf9d9b7cf None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:22.753865 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c088f0d-1fad-48d3-a8f3-53b9411a7396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:22.754432 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c088f0d-1fad-48d3-a8f3-53b9411a7396 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:22.754768 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c088f0d-1fad-48d3-a8f3-53b9411a7396 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:22.755047 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c088f0d-1fad-48d3-a8f3-53b9411a7396 admin admin] PATH_INFO: `/v3/users/0d31b23b1fa648a19110aa8bcddad1df` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:22.756208 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c088f0d-1fad-48d3-a8f3-53b9411a7396 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d31b23b1fa648a19110aa8bcddad1df)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:22.758176 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c088f0d-1fad-48d3-a8f3-53b9411a7396 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:22.906829 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-3c088f0d-1fad-48d3-a8f3-53b9411a7396 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d31b23b1fa648a19110aa8bcddad1df'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:22.907410 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-3c088f0d-1fad-48d3-a8f3-53b9411a7396 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d31b23b1fa648a19110aa8bcddad1df'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:22.908188 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-3c088f0d-1fad-48d3-a8f3-53b9411a7396 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d31b23b1fa648a19110aa8bcddad1df'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:22.978283 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-3c088f0d-1fad-48d3-a8f3-53b9411a7396 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d31b23b1fa648a19110aa8bcddad1df'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:22.991228 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-3c088f0d-1fad-48d3-a8f3-53b9411a7396 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d31b23b1fa648a19110aa8bcddad1df'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:22.993376 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:52:23.005903 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1317/2610] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:52:22 2022] DELETE /identity/v3/users/0d31b23b1fa648a19110aa8bcddad1df => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:52:23.010451 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1074da-322f-4e73-a461-3e4740778cf8 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:23.010941 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1074da-322f-4e73-a461-3e4740778cf8 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:23.011039 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1074da-322f-4e73-a461-3e4740778cf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:23.030605 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-9d1074da-322f-4e73-a461-3e4740778cf8 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:52:23.031817 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-9d1074da-322f-4e73-a461-3e4740778cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:23.094205 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1294/2611] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:52:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:52:23.104648 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c088f0d-1fad-48d3-a8f3-53b9411a7396 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:23.106376 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-3c088f0d-1fad-48d3-a8f3-53b9411a7396 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:23.166593 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c088f0d-1fad-48d3-a8f3-53b9411a7396 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:23.171259 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4076680e-9f6b-49e6-8a22-8eb939a1c428 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:23.172150 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4076680e-9f6b-49e6-8a22-8eb939a1c428 service neutron] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:23.172150 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4076680e-9f6b-49e6-8a22-8eb939a1c428 service neutron] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:23.172535 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4076680e-9f6b-49e6-8a22-8eb939a1c428 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:23.182257 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4076680e-9f6b-49e6-8a22-8eb939a1c428 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:23.183688 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4076680e-9f6b-49e6-8a22-8eb939a1c428 service neutron] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:23.194456 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1318/2612] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:52:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:52:23.620111 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1074da-322f-4e73-a461-3e4740778cf8 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:23.621812 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-9d1074da-322f-4e73-a461-3e4740778cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:23.637319 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1074da-322f-4e73-a461-3e4740778cf8 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:23.651231 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd618a15-9555-4cc1-a095-79eccd0c8193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:23.651231 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd618a15-9555-4cc1-a095-79eccd0c8193 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:23.651231 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd618a15-9555-4cc1-a095-79eccd0c8193 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:23.651231 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd618a15-9555-4cc1-a095-79eccd0c8193 admin admin] PATH_INFO: `/v3/projects/0d92e2cbdd234a91b37f51cf7b31e29d` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:23.651550 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd618a15-9555-4cc1-a095-79eccd0c8193 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d92e2cbdd234a91b37f51cf7b31e29d)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:23.653783 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd618a15-9555-4cc1-a095-79eccd0c8193 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:23.774004 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-bd618a15-9555-4cc1-a095-79eccd0c8193 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d92e2cbdd234a91b37f51cf7b31e29d', 'request_id': 'req-bd618a15-9555-4cc1-a095-79eccd0c8193'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:23.779698 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-bd618a15-9555-4cc1-a095-79eccd0c8193 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d92e2cbdd234a91b37f51cf7b31e29d', 'request_id': 'req-bd618a15-9555-4cc1-a095-79eccd0c8193'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:23.858448 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-bd618a15-9555-4cc1-a095-79eccd0c8193 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d92e2cbdd234a91b37f51cf7b31e29d', 'request_id': 'req-bd618a15-9555-4cc1-a095-79eccd0c8193'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:23.869956 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-bd618a15-9555-4cc1-a095-79eccd0c8193 admin admin] The token cache is being invalidate because project 0d92e2cbdd234a91b37f51cf7b31e29d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:52:23.870242 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-bd618a15-9555-4cc1-a095-79eccd0c8193 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:23.870865 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-bd618a15-9555-4cc1-a095-79eccd0c8193 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:23.872402 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1295/2613] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:52:23 2022] DELETE /identity/v3/projects/0d92e2cbdd234a91b37f51cf7b31e29d => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:52:23.876952 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4076680e-9f6b-49e6-8a22-8eb939a1c428 service neutron] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:23.886837 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4076680e-9f6b-49e6-8a22-8eb939a1c428 service neutron] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:23.946231 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ec2a5f-c240-4151-846b-3bf80e3d9147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:23.946861 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ec2a5f-c240-4151-846b-3bf80e3d9147 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:23.947139 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ec2a5f-c240-4151-846b-3bf80e3d9147 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:23.947417 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ec2a5f-c240-4151-846b-3bf80e3d9147 admin admin] PATH_INFO: `/v3/users/2a7537860cae40f787818766fb5d18ce` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:23.948525 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ec2a5f-c240-4151-846b-3bf80e3d9147 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a7537860cae40f787818766fb5d18ce)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:23.950563 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ec2a5f-c240-4151-846b-3bf80e3d9147 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:24.128524 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-91ec2a5f-c240-4151-846b-3bf80e3d9147 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a7537860cae40f787818766fb5d18ce'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:24.129233 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-91ec2a5f-c240-4151-846b-3bf80e3d9147 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a7537860cae40f787818766fb5d18ce'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:24.129864 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-91ec2a5f-c240-4151-846b-3bf80e3d9147 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a7537860cae40f787818766fb5d18ce'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:24.227779 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-91ec2a5f-c240-4151-846b-3bf80e3d9147 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a7537860cae40f787818766fb5d18ce'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:24.240428 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-91ec2a5f-c240-4151-846b-3bf80e3d9147 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a7537860cae40f787818766fb5d18ce'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:24.254131 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1319/2614] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:52:23 2022] DELETE /identity/v3/users/2a7537860cae40f787818766fb5d18ce => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:52:24.777224 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd618a15-9555-4cc1-a095-79eccd0c8193 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:24.779289 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-bd618a15-9555-4cc1-a095-79eccd0c8193 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:24.797395 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd618a15-9555-4cc1-a095-79eccd0c8193 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:24.861195 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:52:24.877800 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44451681-0e3d-466a-8387-e28bec50b441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:24.878452 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44451681-0e3d-466a-8387-e28bec50b441 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:24.878736 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44451681-0e3d-466a-8387-e28bec50b441 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:24.879018 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44451681-0e3d-466a-8387-e28bec50b441 admin admin] PATH_INFO: `/v3/projects/300eab25dfd747dc87bd60d95b21f4e8` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:24.880153 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44451681-0e3d-466a-8387-e28bec50b441 admin admin] RBAC: Authorizing `identity:delete_project(project_id=300eab25dfd747dc87bd60d95b21f4e8)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:24.882152 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44451681-0e3d-466a-8387-e28bec50b441 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:25.028303 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-44451681-0e3d-466a-8387-e28bec50b441 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '300eab25dfd747dc87bd60d95b21f4e8', 'request_id': 'req-44451681-0e3d-466a-8387-e28bec50b441'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:25.034696 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-44451681-0e3d-466a-8387-e28bec50b441 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '300eab25dfd747dc87bd60d95b21f4e8', 'request_id': 'req-44451681-0e3d-466a-8387-e28bec50b441'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:25.115732 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-44451681-0e3d-466a-8387-e28bec50b441 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '300eab25dfd747dc87bd60d95b21f4e8', 'request_id': 'req-44451681-0e3d-466a-8387-e28bec50b441'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:25.129142 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-44451681-0e3d-466a-8387-e28bec50b441 admin admin] The token cache is being invalidate because project 300eab25dfd747dc87bd60d95b21f4e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:52:25.129469 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-44451681-0e3d-466a-8387-e28bec50b441 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:25.130232 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-44451681-0e3d-466a-8387-e28bec50b441 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:25.131840 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1296/2615] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:52:24 2022] DELETE /identity/v3/projects/300eab25dfd747dc87bd60d95b21f4e8 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:52:25.137384 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcbe832-232d-4418-9af6-11e039b5a0c0 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:25.137895 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcbe832-232d-4418-9af6-11e039b5a0c0 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:25.137997 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcbe832-232d-4418-9af6-11e039b5a0c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:25.159622 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-afcbe832-232d-4418-9af6-11e039b5a0c0 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:52:25.160935 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-afcbe832-232d-4418-9af6-11e039b5a0c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:25.176837 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:52:25.254341 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1320/2616] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:52:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:52:25.260399 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ccbc62-d7ca-4591-b8f7-2369da21fa4f None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:25.260717 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ccbc62-d7ca-4591-b8f7-2369da21fa4f None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:25.260999 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ccbc62-d7ca-4591-b8f7-2369da21fa4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:25.286293 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-50ccbc62-d7ca-4591-b8f7-2369da21fa4f None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:52:25.286293 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-50ccbc62-d7ca-4591-b8f7-2369da21fa4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:25.293376 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1297/2617] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:52:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:52:25.297806 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcbe832-232d-4418-9af6-11e039b5a0c0 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:25.309578 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcbe832-232d-4418-9af6-11e039b5a0c0 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:25.314996 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb1ce82-31c5-4088-b54b-7c41f246664e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:25.315607 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb1ce82-31c5-4088-b54b-7c41f246664e admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:25.315914 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb1ce82-31c5-4088-b54b-7c41f246664e admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:25.316179 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb1ce82-31c5-4088-b54b-7c41f246664e admin admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:25.316864 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb1ce82-31c5-4088-b54b-7c41f246664e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:52:25.317162 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb1ce82-31c5-4088-b54b-7c41f246664e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:25.322519 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb1ce82-31c5-4088-b54b-7c41f246664e admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:25.332774 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1321/2618] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 24 06:52:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:52:25.337392 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ccbc62-d7ca-4591-b8f7-2369da21fa4f None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:25.347488 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ccbc62-d7ca-4591-b8f7-2369da21fa4f None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:25.352560 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1051ea-b0e3-4db2-ba03-4cb34ed99652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:25.357023 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1051ea-b0e3-4db2-ba03-4cb34ed99652 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:25.357023 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1051ea-b0e3-4db2-ba03-4cb34ed99652 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:25.357023 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1051ea-b0e3-4db2-ba03-4cb34ed99652 admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:25.357023 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1051ea-b0e3-4db2-ba03-4cb34ed99652 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:25.357023 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1051ea-b0e3-4db2-ba03-4cb34ed99652 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:25.483717 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1298/2619] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:52:25 2022] POST /identity/v3/projects => generated 377 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:52:25.488679 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb1ce82-31c5-4088-b54b-7c41f246664e admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:25.498694 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb1ce82-31c5-4088-b54b-7c41f246664e admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:25.556033 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637ae21d-a7b5-4f9a-88c9-1e66b5ca7eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:25.556552 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ae21d-a7b5-4f9a-88c9-1e66b5ca7eef admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:25.556830 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ae21d-a7b5-4f9a-88c9-1e66b5ca7eef admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:25.557100 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ae21d-a7b5-4f9a-88c9-1e66b5ca7eef admin admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:25.557760 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637ae21d-a7b5-4f9a-88c9-1e66b5ca7eef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:25.559795 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637ae21d-a7b5-4f9a-88c9-1e66b5ca7eef admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:25.698204 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1322/2620] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:52:25 2022] POST /identity/v3/users => generated 289 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:52:25.704767 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1051ea-b0e3-4db2-ba03-4cb34ed99652 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:25.716667 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1051ea-b0e3-4db2-ba03-4cb34ed99652 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:25.723792 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883db4b5-f14e-4cf2-8d30-d76c3d152897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:25.724930 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883db4b5-f14e-4cf2-8d30-d76c3d152897 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:25.725526 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883db4b5-f14e-4cf2-8d30-d76c3d152897 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:25.725897 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883db4b5-f14e-4cf2-8d30-d76c3d152897 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:25.726593 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883db4b5-f14e-4cf2-8d30-d76c3d152897 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:25.728306 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883db4b5-f14e-4cf2-8d30-d76c3d152897 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:25.739984 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1299/2621] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:52:25 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:52:25.744436 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637ae21d-a7b5-4f9a-88c9-1e66b5ca7eef admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:25.756838 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637ae21d-a7b5-4f9a-88c9-1e66b5ca7eef admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:25.761913 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f72392-1121-4fc4-9c89-45fa7fae90e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:25.762519 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f72392-1121-4fc4-9c89-45fa7fae90e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:25.762801 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f72392-1121-4fc4-9c89-45fa7fae90e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:25.763076 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f72392-1121-4fc4-9c89-45fa7fae90e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:25.763738 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f72392-1121-4fc4-9c89-45fa7fae90e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:25.765844 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f72392-1121-4fc4-9c89-45fa7fae90e3 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:25.777465 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1323/2622] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:52:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:52:25.782443 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883db4b5-f14e-4cf2-8d30-d76c3d152897 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:25.796576 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883db4b5-f14e-4cf2-8d30-d76c3d152897 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:25.804594 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421d4d51-448c-4410-a9bb-8d8fb1d78e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:25.805784 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421d4d51-448c-4410-a9bb-8d8fb1d78e2e admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:25.806315 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421d4d51-448c-4410-a9bb-8d8fb1d78e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:25.806732 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421d4d51-448c-4410-a9bb-8d8fb1d78e2e admin admin] PATH_INFO: `/v3/projects/f780d57f4fb6499bb1ae328da377314d/users/6b2c46d6ed514bc98a586f413f3e39f2/roles/e1f9b167c51c4a388cbc9594db8a465d` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:25.828287 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421d4d51-448c-4410-a9bb-8d8fb1d78e2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f780d57f4fb6499bb1ae328da377314d, user_id=6b2c46d6ed514bc98a586f413f3e39f2, role_id=e1f9b167c51c4a388cbc9594db8a465d)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:25.831037 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421d4d51-448c-4410-a9bb-8d8fb1d78e2e admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:25.943351 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1300/2623] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:52:25 2022] PUT /identity/v3/projects/f780d57f4fb6499bb1ae328da377314d/users/6b2c46d6ed514bc98a586f413f3e39f2/roles/e1f9b167c51c4a388cbc9594db8a465d => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:52:25.959452 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9d94f7-00f2-45eb-b334-ed6a4d9cf791 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:25.959805 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9d94f7-00f2-45eb-b334-ed6a4d9cf791 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:25.960088 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9d94f7-00f2-45eb-b334-ed6a4d9cf791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:25.992181 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-0c9d94f7-00f2-45eb-b334-ed6a4d9cf791 None None] MFA Rules not processed for user `6b2c46d6ed514bc98a586f413f3e39f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:52:25.993456 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-0c9d94f7-00f2-45eb-b334-ed6a4d9cf791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:26.063498 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1324/2624] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:52:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:52:26.068185 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421d4d51-448c-4410-a9bb-8d8fb1d78e2e admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:26.131835 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421d4d51-448c-4410-a9bb-8d8fb1d78e2e admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:26.136667 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e61cb41-4246-464f-8498-a01e3a9048ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:26.137297 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e61cb41-4246-464f-8498-a01e3a9048ca admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:26.137604 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e61cb41-4246-464f-8498-a01e3a9048ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:26.137899 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e61cb41-4246-464f-8498-a01e3a9048ca admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:26.138519 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e61cb41-4246-464f-8498-a01e3a9048ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:26.140967 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e61cb41-4246-464f-8498-a01e3a9048ca admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:26.276450 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:52:26.285045 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1301/2625] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:52:26 2022] POST /identity/v3/projects => generated 377 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:52:26.289315 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9d94f7-00f2-45eb-b334-ed6a4d9cf791 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:26.301706 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9d94f7-00f2-45eb-b334-ed6a4d9cf791 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:26.307266 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:52:26.362117 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f62f54e-b21c-4622-8ef1-019853fab6f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:26.363609 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f62f54e-b21c-4622-8ef1-019853fab6f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:26.363609 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f62f54e-b21c-4622-8ef1-019853fab6f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:26.363609 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f62f54e-b21c-4622-8ef1-019853fab6f1 admin admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:26.363908 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f62f54e-b21c-4622-8ef1-019853fab6f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:26.365795 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f62f54e-b21c-4622-8ef1-019853fab6f1 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:26.451427 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1325/2626] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:52:26 2022] POST /identity/v3/users => generated 295 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:52:26.456892 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e61cb41-4246-464f-8498-a01e3a9048ca admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:26.468834 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e61cb41-4246-464f-8498-a01e3a9048ca admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:26.473640 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42ca869-4f82-42fa-9e91-e7dae13f5512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:26.474255 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42ca869-4f82-42fa-9e91-e7dae13f5512 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:26.474670 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42ca869-4f82-42fa-9e91-e7dae13f5512 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:26.474942 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42ca869-4f82-42fa-9e91-e7dae13f5512 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:26.475588 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42ca869-4f82-42fa-9e91-e7dae13f5512 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:26.477267 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42ca869-4f82-42fa-9e91-e7dae13f5512 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:26.488684 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1302/2627] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:52:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:52:26.493054 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f62f54e-b21c-4622-8ef1-019853fab6f1 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:26.504536 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f62f54e-b21c-4622-8ef1-019853fab6f1 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:26.509595 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319f7bea-d47a-44ad-a157-4d60f2fe28cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:26.510233 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319f7bea-d47a-44ad-a157-4d60f2fe28cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:26.510515 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319f7bea-d47a-44ad-a157-4d60f2fe28cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:26.510791 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319f7bea-d47a-44ad-a157-4d60f2fe28cc admin admin] PATH_INFO: `/v3/projects/a46d605da0744f3d94ea9f2a42358182/users/52011aae6d0f483287d4fa527bbc26f3/roles/486144ad1bcd417e86acdbc1d14327bf` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:26.526901 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319f7bea-d47a-44ad-a157-4d60f2fe28cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=a46d605da0744f3d94ea9f2a42358182, user_id=52011aae6d0f483287d4fa527bbc26f3, role_id=486144ad1bcd417e86acdbc1d14327bf)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:26.529862 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319f7bea-d47a-44ad-a157-4d60f2fe28cc admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:26.625497 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1326/2628] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:52:26 2022] PUT /identity/v3/projects/a46d605da0744f3d94ea9f2a42358182/users/52011aae6d0f483287d4fa527bbc26f3/roles/486144ad1bcd417e86acdbc1d14327bf => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:52:26.629410 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42ca869-4f82-42fa-9e91-e7dae13f5512 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:26.640882 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42ca869-4f82-42fa-9e91-e7dae13f5512 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:26.703142 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43c6085-6468-4781-921a-0238c26928f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:26.703792 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43c6085-6468-4781-921a-0238c26928f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:26.704072 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43c6085-6468-4781-921a-0238c26928f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:26.704467 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43c6085-6468-4781-921a-0238c26928f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:26.705117 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43c6085-6468-4781-921a-0238c26928f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:26.706766 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43c6085-6468-4781-921a-0238c26928f9 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:26.717552 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1303/2629] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:52:26 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:52:26.721870 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319f7bea-d47a-44ad-a157-4d60f2fe28cc admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:26.732889 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319f7bea-d47a-44ad-a157-4d60f2fe28cc admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:26.738401 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ddd3e4-c579-4bac-b223-493a3fb12bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:26.739063 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ddd3e4-c579-4bac-b223-493a3fb12bd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:26.739393 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ddd3e4-c579-4bac-b223-493a3fb12bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:26.739673 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ddd3e4-c579-4bac-b223-493a3fb12bd2 admin admin] PATH_INFO: `/v3/projects/a46d605da0744f3d94ea9f2a42358182/users/52011aae6d0f483287d4fa527bbc26f3/roles/486144ad1bcd417e86acdbc1d14327bf` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:26.741633 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ddd3e4-c579-4bac-b223-493a3fb12bd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a46d605da0744f3d94ea9f2a42358182, user_id=52011aae6d0f483287d4fa527bbc26f3, role_id=486144ad1bcd417e86acdbc1d14327bf)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:26.744258 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ddd3e4-c579-4bac-b223-493a3fb12bd2 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:26.756398 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1327/2630] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:52:26 2022] PUT /identity/v3/projects/a46d605da0744f3d94ea9f2a42358182/users/52011aae6d0f483287d4fa527bbc26f3/roles/486144ad1bcd417e86acdbc1d14327bf => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:52:26.768123 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfde99be-1ced-409a-ace8-19d02d11a4a9 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:26.768414 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfde99be-1ced-409a-ace8-19d02d11a4a9 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:26.768738 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfde99be-1ced-409a-ace8-19d02d11a4a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:26.799169 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-bfde99be-1ced-409a-ace8-19d02d11a4a9 None None] MFA Rules not processed for user `52011aae6d0f483287d4fa527bbc26f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:52:26.800433 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-bfde99be-1ced-409a-ace8-19d02d11a4a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:26.873069 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1304/2631] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:52:26 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:52:26.885017 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ddd3e4-c579-4bac-b223-493a3fb12bd2 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:26.886909 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-39ddd3e4-c579-4bac-b223-493a3fb12bd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:26.951126 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ddd3e4-c579-4bac-b223-493a3fb12bd2 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:26.956338 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b69c50-dc32-4867-b8c2-53332998037f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36639dce4f97412f89d71367f650b901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:26.957060 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b69c50-dc32-4867-b8c2-53332998037f service cinder] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:26.957488 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b69c50-dc32-4867-b8c2-53332998037f service cinder] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:26.957834 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b69c50-dc32-4867-b8c2-53332998037f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:27.024324 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b69c50-dc32-4867-b8c2-53332998037f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:27.026052 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b69c50-dc32-4867-b8c2-53332998037f service cinder] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:27.037286 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1328/2632] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:52:26 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:52:31.025542 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfde99be-1ced-409a-ace8-19d02d11a4a9 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:31.027032 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-bfde99be-1ced-409a-ace8-19d02d11a4a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:31.111161 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfde99be-1ced-409a-ace8-19d02d11a4a9 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:31.117032 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922b4150-6e58-4399-b9ab-96638b70ba7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:31.117708 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922b4150-6e58-4399-b9ab-96638b70ba7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:31.117984 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922b4150-6e58-4399-b9ab-96638b70ba7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:31.118261 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922b4150-6e58-4399-b9ab-96638b70ba7d admin admin] PATH_INFO: `/v3/users/8cbadf33fe01430ca66dcd409ee8eda1` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:31.119499 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922b4150-6e58-4399-b9ab-96638b70ba7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cbadf33fe01430ca66dcd409ee8eda1)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:31.121449 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922b4150-6e58-4399-b9ab-96638b70ba7d admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:31.132780 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:52:31.367506 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-922b4150-6e58-4399-b9ab-96638b70ba7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cbadf33fe01430ca66dcd409ee8eda1'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:31.372554 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-922b4150-6e58-4399-b9ab-96638b70ba7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cbadf33fe01430ca66dcd409ee8eda1'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:31.372554 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-922b4150-6e58-4399-b9ab-96638b70ba7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cbadf33fe01430ca66dcd409ee8eda1'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:31.375557 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-922b4150-6e58-4399-b9ab-96638b70ba7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cbadf33fe01430ca66dcd409ee8eda1'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:31.458237 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-922b4150-6e58-4399-b9ab-96638b70ba7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cbadf33fe01430ca66dcd409ee8eda1'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:31.487643 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1305/2633] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:52:31 2022] DELETE /identity/v3/users/8cbadf33fe01430ca66dcd409ee8eda1 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:52:31.507982 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549523aa-37ea-4b65-a2ab-a828d315f887 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:31.508477 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549523aa-37ea-4b65-a2ab-a828d315f887 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:31.508638 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549523aa-37ea-4b65-a2ab-a828d315f887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:31.529806 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-549523aa-37ea-4b65-a2ab-a828d315f887 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:52:31.530996 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-549523aa-37ea-4b65-a2ab-a828d315f887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:31.546319 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:52:31.604164 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1329/2634] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:52:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:52:31.614091 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922b4150-6e58-4399-b9ab-96638b70ba7d admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:31.615558 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-922b4150-6e58-4399-b9ab-96638b70ba7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:31.678256 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922b4150-6e58-4399-b9ab-96638b70ba7d admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:31.681898 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bebed9-ff4c-4206-9564-9c6f8a8bd182 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:31.682512 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bebed9-ff4c-4206-9564-9c6f8a8bd182 service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:31.682924 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bebed9-ff4c-4206-9564-9c6f8a8bd182 service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:31.683178 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bebed9-ff4c-4206-9564-9c6f8a8bd182 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:31.699392 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bebed9-ff4c-4206-9564-9c6f8a8bd182 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:31.700969 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bebed9-ff4c-4206-9564-9c6f8a8bd182 service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:31.711792 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1306/2635] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:52:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:52:32.212363 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549523aa-37ea-4b65-a2ab-a828d315f887 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:32.213889 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-549523aa-37ea-4b65-a2ab-a828d315f887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:32.231904 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549523aa-37ea-4b65-a2ab-a828d315f887 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:32.232633 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5009019e-5e4a-4eb0-ae2e-4c0d721bff05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:32.233309 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5009019e-5e4a-4eb0-ae2e-4c0d721bff05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:32.233634 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5009019e-5e4a-4eb0-ae2e-4c0d721bff05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:32.233911 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5009019e-5e4a-4eb0-ae2e-4c0d721bff05 admin admin] PATH_INFO: `/v3/projects/6cc9120b955f4804985fb61b6a22ab87` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:32.235136 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5009019e-5e4a-4eb0-ae2e-4c0d721bff05 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6cc9120b955f4804985fb61b6a22ab87)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:32.238198 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5009019e-5e4a-4eb0-ae2e-4c0d721bff05 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:32.385159 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-5009019e-5e4a-4eb0-ae2e-4c0d721bff05 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6cc9120b955f4804985fb61b6a22ab87', 'request_id': 'req-5009019e-5e4a-4eb0-ae2e-4c0d721bff05'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:32.496040 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-5009019e-5e4a-4eb0-ae2e-4c0d721bff05 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6cc9120b955f4804985fb61b6a22ab87', 'request_id': 'req-5009019e-5e4a-4eb0-ae2e-4c0d721bff05'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:32.506665 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-5009019e-5e4a-4eb0-ae2e-4c0d721bff05 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6cc9120b955f4804985fb61b6a22ab87', 'request_id': 'req-5009019e-5e4a-4eb0-ae2e-4c0d721bff05'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:32.513816 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-5009019e-5e4a-4eb0-ae2e-4c0d721bff05 admin admin] The token cache is being invalidate because project 6cc9120b955f4804985fb61b6a22ab87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81723) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:52:32.514411 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-5009019e-5e4a-4eb0-ae2e-4c0d721bff05 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:32.514744 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-5009019e-5e4a-4eb0-ae2e-4c0d721bff05 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:32.516219 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1330/2636] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:52:32 2022] DELETE /identity/v3/projects/6cc9120b955f4804985fb61b6a22ab87 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:52:32.521356 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b0f368-dd3a-4e0b-beaa-a11b9f8791c7 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:32.521819 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b0f368-dd3a-4e0b-beaa-a11b9f8791c7 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:32.521923 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b0f368-dd3a-4e0b-beaa-a11b9f8791c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:32.542280 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-19b0f368-dd3a-4e0b-beaa-a11b9f8791c7 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:52:32.543488 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-19b0f368-dd3a-4e0b-beaa-a11b9f8791c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:32.604049 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1307/2637] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:52:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:52:32.613070 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b222686a-458d-4708-80d2-c90633f4ef3a None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:32.613391 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b222686a-458d-4708-80d2-c90633f4ef3a None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:32.613670 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b222686a-458d-4708-80d2-c90633f4ef3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:32.633604 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-b222686a-458d-4708-80d2-c90633f4ef3a None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:52:32.634845 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-b222686a-458d-4708-80d2-c90633f4ef3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:32.642891 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1331/2638] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:52:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:52:32.647465 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b0f368-dd3a-4e0b-beaa-a11b9f8791c7 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:32.657246 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b0f368-dd3a-4e0b-beaa-a11b9f8791c7 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:32.662864 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16678d96-1eb8-4752-9962-85c0920ffcdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:32.663471 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16678d96-1eb8-4752-9962-85c0920ffcdd admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:32.663748 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16678d96-1eb8-4752-9962-85c0920ffcdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:32.664023 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16678d96-1eb8-4752-9962-85c0920ffcdd admin admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:32.664669 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16678d96-1eb8-4752-9962-85c0920ffcdd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:52:32.664970 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16678d96-1eb8-4752-9962-85c0920ffcdd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:32.666605 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16678d96-1eb8-4752-9962-85c0920ffcdd admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:32.679131 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1308/2639] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 24 06:52:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:52:32.683662 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b222686a-458d-4708-80d2-c90633f4ef3a None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:32.693066 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b222686a-458d-4708-80d2-c90633f4ef3a None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:32.697731 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce82d83-e250-410e-b1b3-3538ae5bb375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:32.698321 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce82d83-e250-410e-b1b3-3538ae5bb375 admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:32.698597 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce82d83-e250-410e-b1b3-3538ae5bb375 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:32.698863 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce82d83-e250-410e-b1b3-3538ae5bb375 admin admin] PATH_INFO: `/v3/projects` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:32.699486 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce82d83-e250-410e-b1b3-3538ae5bb375 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:32.701538 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce82d83-e250-410e-b1b3-3538ae5bb375 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:32.800693 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1332/2640] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:52:32 2022] POST /identity/v3/projects => generated 373 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:52:32.805281 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16678d96-1eb8-4752-9962-85c0920ffcdd admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:32.815327 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16678d96-1eb8-4752-9962-85c0920ffcdd admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:32.827305 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:52:32.872420 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8840a73a-1b8e-489d-b3a6-88ddb6d39ba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:32.873037 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8840a73a-1b8e-489d-b3a6-88ddb6d39ba7 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:32.873350 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8840a73a-1b8e-489d-b3a6-88ddb6d39ba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:32.873659 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8840a73a-1b8e-489d-b3a6-88ddb6d39ba7 admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:32.874321 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8840a73a-1b8e-489d-b3a6-88ddb6d39ba7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:32.876281 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8840a73a-1b8e-489d-b3a6-88ddb6d39ba7 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:33.003823 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1309/2641] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:52:32 2022] POST /identity/v3/users => generated 287 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:52:33.008348 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce82d83-e250-410e-b1b3-3538ae5bb375 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:33.019322 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce82d83-e250-410e-b1b3-3538ae5bb375 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:33.023970 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759fe710-ffec-472b-bc61-1284b9d0c28f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:33.024605 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759fe710-ffec-472b-bc61-1284b9d0c28f admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:33.024887 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759fe710-ffec-472b-bc61-1284b9d0c28f admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:33.025139 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759fe710-ffec-472b-bc61-1284b9d0c28f admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:33.025818 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759fe710-ffec-472b-bc61-1284b9d0c28f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:33.027417 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759fe710-ffec-472b-bc61-1284b9d0c28f admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:33.038113 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1333/2642] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:52:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:52:33.042462 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8840a73a-1b8e-489d-b3a6-88ddb6d39ba7 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:33.056980 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8840a73a-1b8e-489d-b3a6-88ddb6d39ba7 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:33.062344 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11385805-b98b-4351-af80-6e98d0e43d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:33.062952 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11385805-b98b-4351-af80-6e98d0e43d1c admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:33.063234 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11385805-b98b-4351-af80-6e98d0e43d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:33.063505 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11385805-b98b-4351-af80-6e98d0e43d1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:33.064144 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11385805-b98b-4351-af80-6e98d0e43d1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:33.066914 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11385805-b98b-4351-af80-6e98d0e43d1c admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:33.077977 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1310/2643] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:52:33 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:52:33.082556 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759fe710-ffec-472b-bc61-1284b9d0c28f admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:33.092832 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759fe710-ffec-472b-bc61-1284b9d0c28f admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:33.097750 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf73e72-de9c-4c94-8390-9e2e85d777ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:33.098395 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf73e72-de9c-4c94-8390-9e2e85d777ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:33.098861 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf73e72-de9c-4c94-8390-9e2e85d777ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:33.099290 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf73e72-de9c-4c94-8390-9e2e85d777ab admin admin] PATH_INFO: `/v3/projects/0cc00174433b49a2980339c2963671b0/users/2e6e7001705b47a09b6e5d7f08595fc8/roles/e1f9b167c51c4a388cbc9594db8a465d` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:33.115807 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf73e72-de9c-4c94-8390-9e2e85d777ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cc00174433b49a2980339c2963671b0, user_id=2e6e7001705b47a09b6e5d7f08595fc8, role_id=e1f9b167c51c4a388cbc9594db8a465d)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:33.118489 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf73e72-de9c-4c94-8390-9e2e85d777ab admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:33.229863 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1334/2644] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:52:33 2022] PUT /identity/v3/projects/0cc00174433b49a2980339c2963671b0/users/2e6e7001705b47a09b6e5d7f08595fc8/roles/e1f9b167c51c4a388cbc9594db8a465d => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:52:33.235100 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca15182-8f8e-4f1d-a8e0-0d64ae363b8d None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:33.235433 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca15182-8f8e-4f1d-a8e0-0d64ae363b8d None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:33.235712 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca15182-8f8e-4f1d-a8e0-0d64ae363b8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:33.255182 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-0ca15182-8f8e-4f1d-a8e0-0d64ae363b8d None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:52:33.256495 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-0ca15182-8f8e-4f1d-a8e0-0d64ae363b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:33.319514 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1311/2645] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:52:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:52:33.329385 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf73e72-de9c-4c94-8390-9e2e85d777ab admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:33.331297 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-6bf73e72-de9c-4c94-8390-9e2e85d777ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:33.374720 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:52:33.392938 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf73e72-de9c-4c94-8390-9e2e85d777ab admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:33.397512 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb130eb-3fc9-477c-9207-1c4a237eed1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:33.398122 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb130eb-3fc9-477c-9207-1c4a237eed1e service neutron] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:33.398404 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb130eb-3fc9-477c-9207-1c4a237eed1e service neutron] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:33.398679 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb130eb-3fc9-477c-9207-1c4a237eed1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:33.408119 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb130eb-3fc9-477c-9207-1c4a237eed1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:33.409608 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb130eb-3fc9-477c-9207-1c4a237eed1e service neutron] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:33.420980 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1335/2646] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:52:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:52:36.648747 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca15182-8f8e-4f1d-a8e0-0d64ae363b8d None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:36.650613 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-0ca15182-8f8e-4f1d-a8e0-0d64ae363b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:36.745746 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca15182-8f8e-4f1d-a8e0-0d64ae363b8d None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:36.749942 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a137b78-f0ce-43c4-9b24-ed7104326f87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36639dce4f97412f89d71367f650b901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:36.750572 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a137b78-f0ce-43c4-9b24-ed7104326f87 service cinder] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:36.750848 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a137b78-f0ce-43c4-9b24-ed7104326f87 service cinder] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:36.751123 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a137b78-f0ce-43c4-9b24-ed7104326f87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:36.752997 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-7a137b78-f0ce-43c4-9b24-ed7104326f87 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:36.818443 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:52:36.838921 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a137b78-f0ce-43c4-9b24-ed7104326f87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:36.842435 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a137b78-f0ce-43c4-9b24-ed7104326f87 service cinder] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:36.864269 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1312/2647] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:52:36 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:52:40.236226 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5682903b-cc2e-4623-aa0e-83e23a9ab785 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:40.236786 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5682903b-cc2e-4623-aa0e-83e23a9ab785 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:40.236786 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5682903b-cc2e-4623-aa0e-83e23a9ab785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:40.268196 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-5682903b-cc2e-4623-aa0e-83e23a9ab785 None None] MFA Rules not processed for user `2e6e7001705b47a09b6e5d7f08595fc8`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:52:40.269497 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-5682903b-cc2e-4623-aa0e-83e23a9ab785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:40.343631 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1336/2648] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:52:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:52:40.348233 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a137b78-f0ce-43c4-9b24-ed7104326f87 service cinder] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:40.358801 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a137b78-f0ce-43c4-9b24-ed7104326f87 service cinder] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:40.363603 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447ff1d0-894a-46f5-a9f4-8243a0e9b565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:40.364176 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447ff1d0-894a-46f5-a9f4-8243a0e9b565 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:40.364478 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447ff1d0-894a-46f5-a9f4-8243a0e9b565 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:40.364973 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447ff1d0-894a-46f5-a9f4-8243a0e9b565 admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:40.365748 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447ff1d0-894a-46f5-a9f4-8243a0e9b565 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:40.368348 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447ff1d0-894a-46f5-a9f4-8243a0e9b565 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:40.467041 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1313/2649] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:52:40 2022] POST /identity/v3/projects => generated 371 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:52:40.470822 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5682903b-cc2e-4623-aa0e-83e23a9ab785 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:40.483643 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5682903b-cc2e-4623-aa0e-83e23a9ab785 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:40.553088 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e185ee-541e-4c3a-bcb4-7e6142858101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:40.555619 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e185ee-541e-4c3a-bcb4-7e6142858101 admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:40.555906 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e185ee-541e-4c3a-bcb4-7e6142858101 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:40.557374 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e185ee-541e-4c3a-bcb4-7e6142858101 admin admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:40.557685 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e185ee-541e-4c3a-bcb4-7e6142858101 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:40.560287 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e185ee-541e-4c3a-bcb4-7e6142858101 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:40.710795 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1337/2650] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:52:40 2022] POST /identity/v3/users => generated 292 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:52:40.712944 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447ff1d0-894a-46f5-a9f4-8243a0e9b565 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:40.728296 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447ff1d0-894a-46f5-a9f4-8243a0e9b565 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:40.734033 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123e6f12-2ded-4aa8-b297-727063859a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:40.734903 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123e6f12-2ded-4aa8-b297-727063859a91 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:40.735343 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123e6f12-2ded-4aa8-b297-727063859a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:40.735822 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123e6f12-2ded-4aa8-b297-727063859a91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:40.736631 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123e6f12-2ded-4aa8-b297-727063859a91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:40.738395 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123e6f12-2ded-4aa8-b297-727063859a91 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:40.751271 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1314/2651] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:52:40 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:52:40.755585 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e185ee-541e-4c3a-bcb4-7e6142858101 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:40.786654 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e185ee-541e-4c3a-bcb4-7e6142858101 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:40.793272 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbed1b5b-8867-4fb0-820e-db2969a3b416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:40.794073 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbed1b5b-8867-4fb0-820e-db2969a3b416 admin admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:40.794425 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbed1b5b-8867-4fb0-820e-db2969a3b416 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:40.794774 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbed1b5b-8867-4fb0-820e-db2969a3b416 admin admin] PATH_INFO: `/v3/projects/0f64ed78d02b462980c827080ae812a9/users/c8cd28cddb344330bdd7d58031047b4b/roles/486144ad1bcd417e86acdbc1d14327bf` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:40.812800 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbed1b5b-8867-4fb0-820e-db2969a3b416 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f64ed78d02b462980c827080ae812a9, user_id=c8cd28cddb344330bdd7d58031047b4b, role_id=486144ad1bcd417e86acdbc1d14327bf)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:40.816069 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbed1b5b-8867-4fb0-820e-db2969a3b416 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:40.911866 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1338/2652] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:52:40 2022] PUT /identity/v3/projects/0f64ed78d02b462980c827080ae812a9/users/c8cd28cddb344330bdd7d58031047b4b/roles/486144ad1bcd417e86acdbc1d14327bf => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:52:40.915951 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123e6f12-2ded-4aa8-b297-727063859a91 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:40.927243 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123e6f12-2ded-4aa8-b297-727063859a91 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:41.031414 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2639697c-98e9-408f-a52e-28d71197b0a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:41.032039 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2639697c-98e9-408f-a52e-28d71197b0a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:41.032319 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2639697c-98e9-408f-a52e-28d71197b0a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:41.032714 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2639697c-98e9-408f-a52e-28d71197b0a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:41.033399 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2639697c-98e9-408f-a52e-28d71197b0a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:41.035273 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2639697c-98e9-408f-a52e-28d71197b0a8 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:41.049129 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1315/2653] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:52:40 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:52:41.063611 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbed1b5b-8867-4fb0-820e-db2969a3b416 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:41.075177 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbed1b5b-8867-4fb0-820e-db2969a3b416 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:41.081562 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8ea4f2-b675-44c1-bde7-9b2fa7a183a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:41.082248 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8ea4f2-b675-44c1-bde7-9b2fa7a183a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:41.082526 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8ea4f2-b675-44c1-bde7-9b2fa7a183a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:41.082825 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8ea4f2-b675-44c1-bde7-9b2fa7a183a2 admin admin] PATH_INFO: `/v3/projects/0f64ed78d02b462980c827080ae812a9/users/c8cd28cddb344330bdd7d58031047b4b/roles/486144ad1bcd417e86acdbc1d14327bf` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:41.085576 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8ea4f2-b675-44c1-bde7-9b2fa7a183a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f64ed78d02b462980c827080ae812a9, user_id=c8cd28cddb344330bdd7d58031047b4b, role_id=486144ad1bcd417e86acdbc1d14327bf)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:41.088589 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8ea4f2-b675-44c1-bde7-9b2fa7a183a2 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:41.106702 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1339/2654] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:52:41 2022] PUT /identity/v3/projects/0f64ed78d02b462980c827080ae812a9/users/c8cd28cddb344330bdd7d58031047b4b/roles/486144ad1bcd417e86acdbc1d14327bf => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:52:43.852923 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2639697c-98e9-408f-a52e-28d71197b0a8 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:43.855645 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-2639697c-98e9-408f-a52e-28d71197b0a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:43.917990 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2639697c-98e9-408f-a52e-28d71197b0a8 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:43.972300 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc47d97-ec84-46c8-b0cd-70b45a7b3aaf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:43.973533 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc47d97-ec84-46c8-b0cd-70b45a7b3aaf service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:43.973533 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc47d97-ec84-46c8-b0cd-70b45a7b3aaf service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:43.974428 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc47d97-ec84-46c8-b0cd-70b45a7b3aaf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:43.976364 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-5fc47d97-ec84-46c8-b0cd-70b45a7b3aaf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:43.993631 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc47d97-ec84-46c8-b0cd-70b45a7b3aaf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:43.995236 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc47d97-ec84-46c8-b0cd-70b45a7b3aaf service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:44.017408 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1316/2655] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:52:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:52:47.277974 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd6a003-7f27-4125-a7c0-bfccbc603f42 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:47.278503 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd6a003-7f27-4125-a7c0-bfccbc603f42 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:47.278503 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd6a003-7f27-4125-a7c0-bfccbc603f42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:47.313000 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-2fd6a003-7f27-4125-a7c0-bfccbc603f42 None None] MFA Rules not processed for user `c8cd28cddb344330bdd7d58031047b4b`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:52:47.314335 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-2fd6a003-7f27-4125-a7c0-bfccbc603f42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:47.330815 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:52:47.430315 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1340/2656] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:52:47 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:52:47.450043 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc47d97-ec84-46c8-b0cd-70b45a7b3aaf service neutron] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:47.452016 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-5fc47d97-ec84-46c8-b0cd-70b45a7b3aaf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:47.464988 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:52:47.535381 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc47d97-ec84-46c8-b0cd-70b45a7b3aaf service neutron] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:47.540383 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244e798e-10f6-4aa9-8c2e-63db424e663a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4f49c1b08c460f8ba6c451c9c92c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:47.541032 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e798e-10f6-4aa9-8c2e-63db424e663a service nova] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:47.541553 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e798e-10f6-4aa9-8c2e-63db424e663a service nova] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:47.541833 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e798e-10f6-4aa9-8c2e-63db424e663a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:47.601520 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244e798e-10f6-4aa9-8c2e-63db424e663a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:47.603350 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244e798e-10f6-4aa9-8c2e-63db424e663a service nova] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:47.613261 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1317/2657] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:52:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:52:50.196330 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d476d2bb-28a0-4d39-8881-49b2056c2f9b None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:50.202362 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d476d2bb-28a0-4d39-8881-49b2056c2f9b None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:50.202362 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d476d2bb-28a0-4d39-8881-49b2056c2f9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:50.217171 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-d476d2bb-28a0-4d39-8881-49b2056c2f9b None None] MFA Rules not processed for user `2e6e7001705b47a09b6e5d7f08595fc8`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:52:50.218396 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-d476d2bb-28a0-4d39-8881-49b2056c2f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:50.225442 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1341/2658] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:52:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:52:52.992440 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244e798e-10f6-4aa9-8c2e-63db424e663a service nova] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:53.009387 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244e798e-10f6-4aa9-8c2e-63db424e663a service nova] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:53.014626 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977ae073-3743-4f07-8307-0adcbaa5a8c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:53.015700 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977ae073-3743-4f07-8307-0adcbaa5a8c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:53.015988 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977ae073-3743-4f07-8307-0adcbaa5a8c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:53.016381 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977ae073-3743-4f07-8307-0adcbaa5a8c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:53.018172 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-977ae073-3743-4f07-8307-0adcbaa5a8c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:53.099719 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977ae073-3743-4f07-8307-0adcbaa5a8c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:53.101403 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977ae073-3743-4f07-8307-0adcbaa5a8c1 service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:53.111325 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1318/2659] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:52:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:52:56.685503 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d476d2bb-28a0-4d39-8881-49b2056c2f9b None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:56.687118 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-d476d2bb-28a0-4d39-8881-49b2056c2f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:56.776479 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d476d2bb-28a0-4d39-8881-49b2056c2f9b None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:56.784690 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1e5088-e95e-41a3-a2a3-5baf22abc6f3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '306188d42dbd4b5b84082ab3842eaf9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:56.787780 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e5088-e95e-41a3-a2a3-5baf22abc6f3 service placement] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:56.789614 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e5088-e95e-41a3-a2a3-5baf22abc6f3 service placement] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:56.789914 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e5088-e95e-41a3-a2a3-5baf22abc6f3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:56.792086 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-ca1e5088-e95e-41a3-a2a3-5baf22abc6f3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:56.805902 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1e5088-e95e-41a3-a2a3-5baf22abc6f3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:56.808245 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1e5088-e95e-41a3-a2a3-5baf22abc6f3 service placement] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:56.824568 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1342/2660] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:52:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:52:56.830301 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977ae073-3743-4f07-8307-0adcbaa5a8c1 service neutron] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:56.832902 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-977ae073-3743-4f07-8307-0adcbaa5a8c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:56.846256 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977ae073-3743-4f07-8307-0adcbaa5a8c1 service neutron] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:56.850986 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93543e32-6d77-4a88-9c27-822067b16ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:56.852579 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93543e32-6d77-4a88-9c27-822067b16ec4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:56.852864 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93543e32-6d77-4a88-9c27-822067b16ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:56.853137 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93543e32-6d77-4a88-9c27-822067b16ec4 admin admin] PATH_INFO: `/v3/users/52f796d5013d4a31b107fb86c5523c9c` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:56.854854 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93543e32-6d77-4a88-9c27-822067b16ec4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52f796d5013d4a31b107fb86c5523c9c)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:56.856758 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93543e32-6d77-4a88-9c27-822067b16ec4 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:57.063066 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:52:57.073806 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-93543e32-6d77-4a88-9c27-822067b16ec4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52f796d5013d4a31b107fb86c5523c9c'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:57.074458 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-93543e32-6d77-4a88-9c27-822067b16ec4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52f796d5013d4a31b107fb86c5523c9c'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:57.075289 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-93543e32-6d77-4a88-9c27-822067b16ec4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52f796d5013d4a31b107fb86c5523c9c'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:57.082825 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-93543e32-6d77-4a88-9c27-822067b16ec4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52f796d5013d4a31b107fb86c5523c9c'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:57.137172 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-93543e32-6d77-4a88-9c27-822067b16ec4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52f796d5013d4a31b107fb86c5523c9c'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:57.163825 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1319/2661] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:52:56 2022] DELETE /identity/v3/users/52f796d5013d4a31b107fb86c5523c9c => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:52:57.430769 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1e5088-e95e-41a3-a2a3-5baf22abc6f3 service placement] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:57.445408 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1e5088-e95e-41a3-a2a3-5baf22abc6f3 service placement] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:57.518399 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db210e0-2ee2-458e-b856-fc40966523d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:57.519042 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db210e0-2ee2-458e-b856-fc40966523d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:57.520033 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db210e0-2ee2-458e-b856-fc40966523d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:57.520033 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db210e0-2ee2-458e-b856-fc40966523d6 admin admin] PATH_INFO: `/v3/projects/e1bf995c8c544dc29f42e53866cc6bd2` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:57.521383 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db210e0-2ee2-458e-b856-fc40966523d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1bf995c8c544dc29f42e53866cc6bd2)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:57.525425 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db210e0-2ee2-458e-b856-fc40966523d6 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:57.606456 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:52:57.630333 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-8db210e0-2ee2-458e-b856-fc40966523d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1bf995c8c544dc29f42e53866cc6bd2', 'request_id': 'req-8db210e0-2ee2-458e-b856-fc40966523d6'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:57.685769 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-8db210e0-2ee2-458e-b856-fc40966523d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1bf995c8c544dc29f42e53866cc6bd2', 'request_id': 'req-8db210e0-2ee2-458e-b856-fc40966523d6'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:57.692673 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-8db210e0-2ee2-458e-b856-fc40966523d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1bf995c8c544dc29f42e53866cc6bd2', 'request_id': 'req-8db210e0-2ee2-458e-b856-fc40966523d6'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:57.704351 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-8db210e0-2ee2-458e-b856-fc40966523d6 admin admin] The token cache is being invalidate because project e1bf995c8c544dc29f42e53866cc6bd2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81723) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:52:57.704638 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-8db210e0-2ee2-458e-b856-fc40966523d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:57.705711 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-8db210e0-2ee2-458e-b856-fc40966523d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:52:57.707218 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1343/2662] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:52:57 2022] DELETE /identity/v3/projects/e1bf995c8c544dc29f42e53866cc6bd2 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:52:57.712171 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af560567-4dc6-4b37-9e14-57b0fa9cd7a5 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:57.712459 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af560567-4dc6-4b37-9e14-57b0fa9cd7a5 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:57.712846 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af560567-4dc6-4b37-9e14-57b0fa9cd7a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:57.733918 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-af560567-4dc6-4b37-9e14-57b0fa9cd7a5 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:52:57.735150 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-af560567-4dc6-4b37-9e14-57b0fa9cd7a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:57.801150 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1320/2663] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:52:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:52:57.810070 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb88b96c-312b-4615-8e63-c2f09880d3d1 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:57.810349 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb88b96c-312b-4615-8e63-c2f09880d3d1 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:57.810629 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb88b96c-312b-4615-8e63-c2f09880d3d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:57.830978 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-bb88b96c-312b-4615-8e63-c2f09880d3d1 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:52:57.832183 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-bb88b96c-312b-4615-8e63-c2f09880d3d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:57.840949 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1344/2664] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:52:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:52:57.845298 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af560567-4dc6-4b37-9e14-57b0fa9cd7a5 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:57.856101 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af560567-4dc6-4b37-9e14-57b0fa9cd7a5 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:57.860814 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ee9779-3752-4f48-af71-4d458d6feccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:57.861451 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ee9779-3752-4f48-af71-4d458d6feccb admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:57.861729 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ee9779-3752-4f48-af71-4d458d6feccb admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:57.861999 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ee9779-3752-4f48-af71-4d458d6feccb admin admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:57.862652 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ee9779-3752-4f48-af71-4d458d6feccb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:52:57.862955 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ee9779-3752-4f48-af71-4d458d6feccb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:57.864542 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ee9779-3752-4f48-af71-4d458d6feccb admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:57.876923 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1321/2665] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 24 06:52:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:52:57.881435 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb88b96c-312b-4615-8e63-c2f09880d3d1 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:57.891271 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb88b96c-312b-4615-8e63-c2f09880d3d1 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:57.896172 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69af71ca-c50d-4177-a104-60a73a70ef69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:57.896790 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69af71ca-c50d-4177-a104-60a73a70ef69 admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:57.897061 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69af71ca-c50d-4177-a104-60a73a70ef69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:57.897366 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69af71ca-c50d-4177-a104-60a73a70ef69 admin admin] PATH_INFO: `/v3/projects` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:57.897996 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69af71ca-c50d-4177-a104-60a73a70ef69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:57.899927 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69af71ca-c50d-4177-a104-60a73a70ef69 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:58.006551 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1345/2666] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:52:57 2022] POST /identity/v3/projects => generated 383 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:52:58.011324 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ee9779-3752-4f48-af71-4d458d6feccb admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:58.021797 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ee9779-3752-4f48-af71-4d458d6feccb admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:58.082179 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9e4533-d308-41e1-a10c-405b0ec57995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:58.082792 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e4533-d308-41e1-a10c-405b0ec57995 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:58.083073 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e4533-d308-41e1-a10c-405b0ec57995 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:58.083345 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e4533-d308-41e1-a10c-405b0ec57995 admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:58.083958 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9e4533-d308-41e1-a10c-405b0ec57995 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:58.085914 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9e4533-d308-41e1-a10c-405b0ec57995 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:58.201946 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1322/2667] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:52:58 2022] POST /identity/v3/users => generated 292 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:52:58.205944 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69af71ca-c50d-4177-a104-60a73a70ef69 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:58.218544 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69af71ca-c50d-4177-a104-60a73a70ef69 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:58.223140 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5d06e0-1ad9-48f2-b8b1-17538a8bb0bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:58.223742 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5d06e0-1ad9-48f2-b8b1-17538a8bb0bf admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:58.224020 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5d06e0-1ad9-48f2-b8b1-17538a8bb0bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:58.224290 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5d06e0-1ad9-48f2-b8b1-17538a8bb0bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:58.224982 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5d06e0-1ad9-48f2-b8b1-17538a8bb0bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:58.226648 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5d06e0-1ad9-48f2-b8b1-17538a8bb0bf admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:58.237388 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1346/2668] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:52:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:52:58.241460 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9e4533-d308-41e1-a10c-405b0ec57995 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:58.251780 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9e4533-d308-41e1-a10c-405b0ec57995 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:58.256628 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fcdb61-3376-4ced-9ad0-58cd213824bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:58.257196 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fcdb61-3376-4ced-9ad0-58cd213824bf admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:58.257491 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fcdb61-3376-4ced-9ad0-58cd213824bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:58.257763 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fcdb61-3376-4ced-9ad0-58cd213824bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:58.258405 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fcdb61-3376-4ced-9ad0-58cd213824bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:58.260536 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fcdb61-3376-4ced-9ad0-58cd213824bf admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:58.270353 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1323/2669] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:52:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:52:58.274404 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5d06e0-1ad9-48f2-b8b1-17538a8bb0bf admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:58.285509 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5d06e0-1ad9-48f2-b8b1-17538a8bb0bf admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:58.290206 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51487935-9389-4dbc-9176-1fe9cf23425c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:58.290913 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51487935-9389-4dbc-9176-1fe9cf23425c admin admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:58.291191 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51487935-9389-4dbc-9176-1fe9cf23425c admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:58.291591 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51487935-9389-4dbc-9176-1fe9cf23425c admin admin] PATH_INFO: `/v3/projects/2689efdab9f440b5a928a6ce643c4f69/users/672493753ec34139a94535e528896165/roles/e1f9b167c51c4a388cbc9594db8a465d` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:58.309819 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51487935-9389-4dbc-9176-1fe9cf23425c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2689efdab9f440b5a928a6ce643c4f69, user_id=672493753ec34139a94535e528896165, role_id=e1f9b167c51c4a388cbc9594db8a465d)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:58.312482 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51487935-9389-4dbc-9176-1fe9cf23425c admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:58.407009 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1347/2670] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:52:58 2022] PUT /identity/v3/projects/2689efdab9f440b5a928a6ce643c4f69/users/672493753ec34139a94535e528896165/roles/e1f9b167c51c4a388cbc9594db8a465d => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:52:58.418497 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca6edd8-daa5-491c-b54b-849f7bf4972d None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:58.418789 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca6edd8-daa5-491c-b54b-849f7bf4972d None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:58.419067 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca6edd8-daa5-491c-b54b-849f7bf4972d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:58.448922 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-9ca6edd8-daa5-491c-b54b-849f7bf4972d None None] MFA Rules not processed for user `672493753ec34139a94535e528896165`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:52:58.450241 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-9ca6edd8-daa5-491c-b54b-849f7bf4972d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:58.455417 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:52:58.519672 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1324/2671] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:52:58 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:52:58.532921 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51487935-9389-4dbc-9176-1fe9cf23425c admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:52:58.534949 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-51487935-9389-4dbc-9176-1fe9cf23425c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:52:58.623686 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51487935-9389-4dbc-9176-1fe9cf23425c admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:52:58.629678 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cfa721-44f2-4a10-bd05-f96a8951ea9f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4f49c1b08c460f8ba6c451c9c92c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:52:58.630294 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cfa721-44f2-4a10-bd05-f96a8951ea9f service nova] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:52:58.630575 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cfa721-44f2-4a10-bd05-f96a8951ea9f service nova] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:52:58.630847 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cfa721-44f2-4a10-bd05-f96a8951ea9f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:52:58.640147 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cfa721-44f2-4a10-bd05-f96a8951ea9f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:52:58.641780 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cfa721-44f2-4a10-bd05-f96a8951ea9f service nova] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:52:58.651068 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1348/2672] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:52:58 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:53:08.561615 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=24 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:53:08.646622 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=74,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:53:17.472847 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca6edd8-daa5-491c-b54b-849f7bf4972d None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:17.547265 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca6edd8-daa5-491c-b54b-849f7bf4972d None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:17.553535 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d091f3a-92c8-4095-ae98-571ca6f81bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:17.554227 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d091f3a-92c8-4095-ae98-571ca6f81bc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:17.554497 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d091f3a-92c8-4095-ae98-571ca6f81bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:17.555733 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d091f3a-92c8-4095-ae98-571ca6f81bc0 admin admin] PATH_INFO: `/v3/users/672493753ec34139a94535e528896165` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:17.555912 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d091f3a-92c8-4095-ae98-571ca6f81bc0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=672493753ec34139a94535e528896165)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:17.558579 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d091f3a-92c8-4095-ae98-571ca6f81bc0 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:17.752869 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-0d091f3a-92c8-4095-ae98-571ca6f81bc0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '672493753ec34139a94535e528896165'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:17.753568 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-0d091f3a-92c8-4095-ae98-571ca6f81bc0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '672493753ec34139a94535e528896165'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:17.754174 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-0d091f3a-92c8-4095-ae98-571ca6f81bc0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '672493753ec34139a94535e528896165'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:17.761504 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-0d091f3a-92c8-4095-ae98-571ca6f81bc0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '672493753ec34139a94535e528896165'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:17.827953 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-0d091f3a-92c8-4095-ae98-571ca6f81bc0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '672493753ec34139a94535e528896165'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:17.858263 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1325/2673] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:53:17 2022] DELETE /identity/v3/users/672493753ec34139a94535e528896165 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:53:17.858489 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dbe5cf-05bb-4608-8f51-3d7c85ac9143 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:17.858489 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dbe5cf-05bb-4608-8f51-3d7c85ac9143 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:17.858489 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dbe5cf-05bb-4608-8f51-3d7c85ac9143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:17.875613 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-07dbe5cf-05bb-4608-8f51-3d7c85ac9143 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:53:17.876808 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-07dbe5cf-05bb-4608-8f51-3d7c85ac9143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:17.937029 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1349/2674] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:53:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:53:17.952441 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d091f3a-92c8-4095-ae98-571ca6f81bc0 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:17.952441 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-0d091f3a-92c8-4095-ae98-571ca6f81bc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:18.006470 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d091f3a-92c8-4095-ae98-571ca6f81bc0 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:18.010101 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02250585-eaba-482f-b860-61b7d096c945 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:18.010736 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02250585-eaba-482f-b860-61b7d096c945 service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:18.011014 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02250585-eaba-482f-b860-61b7d096c945 service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:18.011284 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02250585-eaba-482f-b860-61b7d096c945 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:18.020248 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02250585-eaba-482f-b860-61b7d096c945 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:18.021743 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02250585-eaba-482f-b860-61b7d096c945 service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:18.031218 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1326/2675] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:53:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:53:18.464650 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dbe5cf-05bb-4608-8f51-3d7c85ac9143 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:18.466117 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-07dbe5cf-05bb-4608-8f51-3d7c85ac9143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:18.479986 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dbe5cf-05bb-4608-8f51-3d7c85ac9143 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:18.485172 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cf2139-9887-468d-a107-ca0264df7ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:18.485868 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cf2139-9887-468d-a107-ca0264df7ea1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:18.486144 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cf2139-9887-468d-a107-ca0264df7ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:18.486416 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cf2139-9887-468d-a107-ca0264df7ea1 admin admin] PATH_INFO: `/v3/projects/2689efdab9f440b5a928a6ce643c4f69` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:18.488328 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cf2139-9887-468d-a107-ca0264df7ea1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2689efdab9f440b5a928a6ce643c4f69)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:18.490403 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cf2139-9887-468d-a107-ca0264df7ea1 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:18.634650 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-41cf2139-9887-468d-a107-ca0264df7ea1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2689efdab9f440b5a928a6ce643c4f69', 'request_id': 'req-41cf2139-9887-468d-a107-ca0264df7ea1'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:18.696732 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-41cf2139-9887-468d-a107-ca0264df7ea1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2689efdab9f440b5a928a6ce643c4f69', 'request_id': 'req-41cf2139-9887-468d-a107-ca0264df7ea1'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:18.706256 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-41cf2139-9887-468d-a107-ca0264df7ea1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2689efdab9f440b5a928a6ce643c4f69', 'request_id': 'req-41cf2139-9887-468d-a107-ca0264df7ea1'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:18.717977 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-41cf2139-9887-468d-a107-ca0264df7ea1 admin admin] The token cache is being invalidate because project 2689efdab9f440b5a928a6ce643c4f69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81723) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:53:18.718267 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-41cf2139-9887-468d-a107-ca0264df7ea1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:18.718924 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-41cf2139-9887-468d-a107-ca0264df7ea1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:18.720419 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1350/2676] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:53:18 2022] DELETE /identity/v3/projects/2689efdab9f440b5a928a6ce643c4f69 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:53:18.725410 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28786458-8bae-4c2e-bd2d-3c111baf8607 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:18.725944 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28786458-8bae-4c2e-bd2d-3c111baf8607 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:18.726050 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28786458-8bae-4c2e-bd2d-3c111baf8607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:18.746663 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-28786458-8bae-4c2e-bd2d-3c111baf8607 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:53:18.747945 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-28786458-8bae-4c2e-bd2d-3c111baf8607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:18.766790 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:53:18.812715 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1327/2677] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:53:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:53:18.821327 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25a5f0d-ce99-4084-8afb-dade22ce359a None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:18.821617 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25a5f0d-ce99-4084-8afb-dade22ce359a None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:18.821890 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25a5f0d-ce99-4084-8afb-dade22ce359a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:18.847556 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-e25a5f0d-ce99-4084-8afb-dade22ce359a None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:53:18.848765 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-e25a5f0d-ce99-4084-8afb-dade22ce359a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:18.857274 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1351/2678] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:53:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:53:18.861556 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28786458-8bae-4c2e-bd2d-3c111baf8607 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:18.871840 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28786458-8bae-4c2e-bd2d-3c111baf8607 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:18.878856 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d748b10-f1cb-47bb-8f63-67f9bb90b9ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:18.879464 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d748b10-f1cb-47bb-8f63-67f9bb90b9ad admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:18.880650 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d748b10-f1cb-47bb-8f63-67f9bb90b9ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:18.880650 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d748b10-f1cb-47bb-8f63-67f9bb90b9ad admin admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:18.880854 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d748b10-f1cb-47bb-8f63-67f9bb90b9ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:53:18.881029 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d748b10-f1cb-47bb-8f63-67f9bb90b9ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:18.882818 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d748b10-f1cb-47bb-8f63-67f9bb90b9ad admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:18.895528 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1328/2679] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 24 06:53:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:53:18.899865 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25a5f0d-ce99-4084-8afb-dade22ce359a None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:18.909517 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25a5f0d-ce99-4084-8afb-dade22ce359a None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:18.914408 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93379e33-d5d8-4f22-b869-e5a286854678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:18.915084 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93379e33-d5d8-4f22-b869-e5a286854678 admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:18.915360 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93379e33-d5d8-4f22-b869-e5a286854678 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:18.915660 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93379e33-d5d8-4f22-b869-e5a286854678 admin admin] PATH_INFO: `/v3/projects` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:18.916292 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93379e33-d5d8-4f22-b869-e5a286854678 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:18.918422 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93379e33-d5d8-4f22-b869-e5a286854678 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:19.035982 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1352/2680] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:53:18 2022] POST /identity/v3/projects => generated 403 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:53:19.040263 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d748b10-f1cb-47bb-8f63-67f9bb90b9ad admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:19.052404 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d748b10-f1cb-47bb-8f63-67f9bb90b9ad admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:19.109862 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4f4f64-7db0-4a63-bc9d-f74ee128445e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:19.110472 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4f4f64-7db0-4a63-bc9d-f74ee128445e admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:19.110752 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4f4f64-7db0-4a63-bc9d-f74ee128445e admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:19.111024 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4f4f64-7db0-4a63-bc9d-f74ee128445e admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:19.111703 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4f4f64-7db0-4a63-bc9d-f74ee128445e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:19.113651 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4f4f64-7db0-4a63-bc9d-f74ee128445e admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:19.233845 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1329/2681] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:53:19 2022] POST /identity/v3/users => generated 302 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:53:19.238505 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93379e33-d5d8-4f22-b869-e5a286854678 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:19.253100 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93379e33-d5d8-4f22-b869-e5a286854678 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:19.259035 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad97568-1c36-4244-91e8-50977d88d336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:19.259785 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad97568-1c36-4244-91e8-50977d88d336 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:19.260222 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad97568-1c36-4244-91e8-50977d88d336 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:19.261101 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad97568-1c36-4244-91e8-50977d88d336 admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:19.261495 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad97568-1c36-4244-91e8-50977d88d336 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:19.263591 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad97568-1c36-4244-91e8-50977d88d336 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:19.277953 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1353/2682] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:53:19 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:53:19.282330 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4f4f64-7db0-4a63-bc9d-f74ee128445e admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:19.293047 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4f4f64-7db0-4a63-bc9d-f74ee128445e admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:19.297977 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dcec8e-c46f-4112-8dff-f160eeb85970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:19.299239 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dcec8e-c46f-4112-8dff-f160eeb85970 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:19.299239 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dcec8e-c46f-4112-8dff-f160eeb85970 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:19.299239 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dcec8e-c46f-4112-8dff-f160eeb85970 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:19.299719 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dcec8e-c46f-4112-8dff-f160eeb85970 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:19.301332 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dcec8e-c46f-4112-8dff-f160eeb85970 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:19.313616 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1330/2683] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:53:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:53:19.318054 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad97568-1c36-4244-91e8-50977d88d336 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:19.328263 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad97568-1c36-4244-91e8-50977d88d336 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:19.333158 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab61aea-d54d-4726-8c2b-a7ca7e94b3e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:19.337942 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab61aea-d54d-4726-8c2b-a7ca7e94b3e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:19.337942 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab61aea-d54d-4726-8c2b-a7ca7e94b3e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:19.337942 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab61aea-d54d-4726-8c2b-a7ca7e94b3e2 admin admin] PATH_INFO: `/v3/projects/153fb2e409044a1f8aa05c7cdb3979db/users/b98979779f414c4e82f24d030ca98197/roles/e1f9b167c51c4a388cbc9594db8a465d` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:19.350700 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab61aea-d54d-4726-8c2b-a7ca7e94b3e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=153fb2e409044a1f8aa05c7cdb3979db, user_id=b98979779f414c4e82f24d030ca98197, role_id=e1f9b167c51c4a388cbc9594db8a465d)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:19.353504 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab61aea-d54d-4726-8c2b-a7ca7e94b3e2 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:19.440480 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1354/2684] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:53:19 2022] PUT /identity/v3/projects/153fb2e409044a1f8aa05c7cdb3979db/users/b98979779f414c4e82f24d030ca98197/roles/e1f9b167c51c4a388cbc9594db8a465d => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:53:19.445613 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd192d17-f8d2-4dda-a602-a6b336f74987 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:19.445894 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd192d17-f8d2-4dda-a602-a6b336f74987 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:19.446167 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd192d17-f8d2-4dda-a602-a6b336f74987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:19.465754 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-bd192d17-f8d2-4dda-a602-a6b336f74987 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:53:19.466957 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-bd192d17-f8d2-4dda-a602-a6b336f74987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:19.533083 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1331/2685] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:53:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:53:19.543403 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab61aea-d54d-4726-8c2b-a7ca7e94b3e2 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:19.545349 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-9ab61aea-d54d-4726-8c2b-a7ca7e94b3e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:19.568286 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:53:19.610155 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab61aea-d54d-4726-8c2b-a7ca7e94b3e2 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:19.613979 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6b1319-c1aa-40d5-aadd-5adcbf3f07e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:19.614595 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6b1319-c1aa-40d5-aadd-5adcbf3f07e7 service neutron] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:19.614874 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6b1319-c1aa-40d5-aadd-5adcbf3f07e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:19.615152 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6b1319-c1aa-40d5-aadd-5adcbf3f07e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:19.629881 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6b1319-c1aa-40d5-aadd-5adcbf3f07e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:19.633542 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6b1319-c1aa-40d5-aadd-5adcbf3f07e7 service neutron] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:19.641834 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1355/2686] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:53:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:53:20.086923 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd192d17-f8d2-4dda-a602-a6b336f74987 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:20.089448 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-bd192d17-f8d2-4dda-a602-a6b336f74987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:20.102169 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd192d17-f8d2-4dda-a602-a6b336f74987 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:20.106867 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87d3537-0aff-4e6f-b4e7-dff83ab1bb84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:20.107618 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87d3537-0aff-4e6f-b4e7-dff83ab1bb84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:20.107919 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87d3537-0aff-4e6f-b4e7-dff83ab1bb84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:20.108188 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87d3537-0aff-4e6f-b4e7-dff83ab1bb84 admin admin] PATH_INFO: `/v3/users/6b2c46d6ed514bc98a586f413f3e39f2` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:20.109363 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87d3537-0aff-4e6f-b4e7-dff83ab1bb84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b2c46d6ed514bc98a586f413f3e39f2)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:20.111287 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87d3537-0aff-4e6f-b4e7-dff83ab1bb84 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:20.140502 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:53:20.294083 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-c87d3537-0aff-4e6f-b4e7-dff83ab1bb84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b2c46d6ed514bc98a586f413f3e39f2'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:20.294911 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-c87d3537-0aff-4e6f-b4e7-dff83ab1bb84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b2c46d6ed514bc98a586f413f3e39f2'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:20.295547 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-c87d3537-0aff-4e6f-b4e7-dff83ab1bb84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b2c46d6ed514bc98a586f413f3e39f2'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:20.303582 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-c87d3537-0aff-4e6f-b4e7-dff83ab1bb84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b2c46d6ed514bc98a586f413f3e39f2'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:20.404850 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-c87d3537-0aff-4e6f-b4e7-dff83ab1bb84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b2c46d6ed514bc98a586f413f3e39f2'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:20.430332 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1332/2687] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:53:20 2022] DELETE /identity/v3/users/6b2c46d6ed514bc98a586f413f3e39f2 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:53:20.436097 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc01729-3721-4ad8-b0b8-05665e3546ab None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:20.436775 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc01729-3721-4ad8-b0b8-05665e3546ab None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:20.436775 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc01729-3721-4ad8-b0b8-05665e3546ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:20.457433 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-0bc01729-3721-4ad8-b0b8-05665e3546ab None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:53:20.458674 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-0bc01729-3721-4ad8-b0b8-05665e3546ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:20.523331 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1356/2688] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:53:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:53:20.534427 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87d3537-0aff-4e6f-b4e7-dff83ab1bb84 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:20.536158 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-c87d3537-0aff-4e6f-b4e7-dff83ab1bb84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:20.598485 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87d3537-0aff-4e6f-b4e7-dff83ab1bb84 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:20.602128 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4533515-6927-4d58-9406-9c605d1e09d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:20.602800 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4533515-6927-4d58-9406-9c605d1e09d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:20.603036 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4533515-6927-4d58-9406-9c605d1e09d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:20.603305 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4533515-6927-4d58-9406-9c605d1e09d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:20.615506 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4533515-6927-4d58-9406-9c605d1e09d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:20.620396 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4533515-6927-4d58-9406-9c605d1e09d3 service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:20.629800 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1333/2689] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:53:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:53:20.842334 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a1eb9a-6a17-4cda-8517-e1c28ec02e5b None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:20.842601 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a1eb9a-6a17-4cda-8517-e1c28ec02e5b None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:20.842978 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a1eb9a-6a17-4cda-8517-e1c28ec02e5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:20.875631 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-a1a1eb9a-6a17-4cda-8517-e1c28ec02e5b None None] MFA Rules not processed for user `b98979779f414c4e82f24d030ca98197`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:53:20.876835 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-a1a1eb9a-6a17-4cda-8517-e1c28ec02e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:20.954353 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1357/2690] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:53:20 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:53:20.958991 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4533515-6927-4d58-9406-9c605d1e09d3 service neutron] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:20.960644 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-f4533515-6927-4d58-9406-9c605d1e09d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:20.974581 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4533515-6927-4d58-9406-9c605d1e09d3 service neutron] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:20.980713 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb42ac88-9fe8-4761-9e25-dbd5057c8af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:20.981368 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb42ac88-9fe8-4761-9e25-dbd5057c8af6 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:20.981645 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb42ac88-9fe8-4761-9e25-dbd5057c8af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:20.981917 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb42ac88-9fe8-4761-9e25-dbd5057c8af6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:20.982580 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb42ac88-9fe8-4761-9e25-dbd5057c8af6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:20.984528 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb42ac88-9fe8-4761-9e25-dbd5057c8af6 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:21.076259 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a1eb9a-6a17-4cda-8517-e1c28ec02e5b None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:21.078036 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-a1a1eb9a-6a17-4cda-8517-e1c28ec02e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:21.095119 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a1eb9a-6a17-4cda-8517-e1c28ec02e5b None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:21.109364 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1334/2691] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:53:20 2022] POST /identity/v3/projects => generated 405 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:53:21.110035 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67681195-4599-4ade-9055-4ced048266f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:21.110035 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67681195-4599-4ade-9055-4ced048266f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:21.110035 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67681195-4599-4ade-9055-4ced048266f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:21.110035 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67681195-4599-4ade-9055-4ced048266f4 admin admin] PATH_INFO: `/v3/projects/f780d57f4fb6499bb1ae328da377314d` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:21.110371 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb42ac88-9fe8-4761-9e25-dbd5057c8af6 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:21.112958 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67681195-4599-4ade-9055-4ced048266f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f780d57f4fb6499bb1ae328da377314d)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:21.114962 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67681195-4599-4ade-9055-4ced048266f4 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:21.181598 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb42ac88-9fe8-4761-9e25-dbd5057c8af6 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:21.188585 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54feb9b3-1385-44ca-a7ee-59d98840ab46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:21.189302 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54feb9b3-1385-44ca-a7ee-59d98840ab46 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:21.189616 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54feb9b3-1385-44ca-a7ee-59d98840ab46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:21.190012 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54feb9b3-1385-44ca-a7ee-59d98840ab46 admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:21.190619 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54feb9b3-1385-44ca-a7ee-59d98840ab46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:21.192572 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54feb9b3-1385-44ca-a7ee-59d98840ab46 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:21.233629 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:53:21.235104 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-67681195-4599-4ade-9055-4ced048266f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f780d57f4fb6499bb1ae328da377314d', 'request_id': 'req-67681195-4599-4ade-9055-4ced048266f4'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:21.301441 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1335/2692] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:53:21 2022] POST /identity/v3/users => generated 309 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:53:21.305468 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54feb9b3-1385-44ca-a7ee-59d98840ab46 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:21.316524 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54feb9b3-1385-44ca-a7ee-59d98840ab46 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:21.325257 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=89,keystone:DELETE=6,keystone:INSERT=5 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:53:21.327123 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-67681195-4599-4ade-9055-4ced048266f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f780d57f4fb6499bb1ae328da377314d', 'request_id': 'req-67681195-4599-4ade-9055-4ced048266f4'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:21.335557 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-67681195-4599-4ade-9055-4ced048266f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f780d57f4fb6499bb1ae328da377314d', 'request_id': 'req-67681195-4599-4ade-9055-4ced048266f4'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:21.347662 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-67681195-4599-4ade-9055-4ced048266f4 admin admin] The token cache is being invalidate because project f780d57f4fb6499bb1ae328da377314d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81723) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:53:21.347907 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-67681195-4599-4ade-9055-4ced048266f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:21.348576 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-67681195-4599-4ade-9055-4ced048266f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:21.350131 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1358/2693] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:53:21 2022] DELETE /identity/v3/projects/f780d57f4fb6499bb1ae328da377314d => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:53:21.354099 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67681195-4599-4ade-9055-4ced048266f4 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:21.359044 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-67681195-4599-4ade-9055-4ced048266f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:21.378853 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a1bffa-7de0-402e-aba0-819487f4c3dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:21.379470 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a1bffa-7de0-402e-aba0-819487f4c3dc admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:21.379745 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a1bffa-7de0-402e-aba0-819487f4c3dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:21.381751 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a1bffa-7de0-402e-aba0-819487f4c3dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:21.381751 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a1bffa-7de0-402e-aba0-819487f4c3dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:21.382290 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a1bffa-7de0-402e-aba0-819487f4c3dc admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:21.392807 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1336/2694] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:53:21 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:53:21.397148 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a1bffa-7de0-402e-aba0-819487f4c3dc admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:21.398927 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-d2a1bffa-7de0-402e-aba0-819487f4c3dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:21.411799 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a1bffa-7de0-402e-aba0-819487f4c3dc admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:21.417391 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d0536a-8941-4498-bd6b-dfa6d445eaa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:21.418309 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d0536a-8941-4498-bd6b-dfa6d445eaa1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:21.418588 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d0536a-8941-4498-bd6b-dfa6d445eaa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:21.418868 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d0536a-8941-4498-bd6b-dfa6d445eaa1 admin admin] PATH_INFO: `/v3/projects/b5d0cf164399493e8ac5c0c854478a74/users/9a8b658af1ab4ce8a169183a61a5f8d9/roles/486144ad1bcd417e86acdbc1d14327bf` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:21.434683 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67681195-4599-4ade-9055-4ced048266f4 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:21.434972 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d0536a-8941-4498-bd6b-dfa6d445eaa1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5d0cf164399493e8ac5c0c854478a74, user_id=9a8b658af1ab4ce8a169183a61a5f8d9, role_id=486144ad1bcd417e86acdbc1d14327bf)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:21.437600 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d0536a-8941-4498-bd6b-dfa6d445eaa1 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:21.442485 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03924c5-5906-4bcd-8b95-728d5aefe5a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:21.446158 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03924c5-5906-4bcd-8b95-728d5aefe5a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:21.446158 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03924c5-5906-4bcd-8b95-728d5aefe5a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:21.446158 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03924c5-5906-4bcd-8b95-728d5aefe5a5 admin admin] PATH_INFO: `/v3/users/52011aae6d0f483287d4fa527bbc26f3` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:21.446158 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03924c5-5906-4bcd-8b95-728d5aefe5a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52011aae6d0f483287d4fa527bbc26f3)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:21.448191 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03924c5-5906-4bcd-8b95-728d5aefe5a5 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:21.497813 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1337/2695] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:53:21 2022] PUT /identity/v3/projects/b5d0cf164399493e8ac5c0c854478a74/users/9a8b658af1ab4ce8a169183a61a5f8d9/roles/486144ad1bcd417e86acdbc1d14327bf => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:53:21.501947 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d0536a-8941-4498-bd6b-dfa6d445eaa1 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:21.563994 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d0536a-8941-4498-bd6b-dfa6d445eaa1 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:21.568704 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99213e39-eca9-414a-8184-c4daac916e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:21.569340 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99213e39-eca9-414a-8184-c4daac916e30 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:21.569639 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99213e39-eca9-414a-8184-c4daac916e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:21.569900 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99213e39-eca9-414a-8184-c4daac916e30 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:21.570533 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99213e39-eca9-414a-8184-c4daac916e30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:21.572145 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99213e39-eca9-414a-8184-c4daac916e30 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:21.582345 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1338/2696] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:53:21 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:53:21.586575 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99213e39-eca9-414a-8184-c4daac916e30 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:21.596561 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99213e39-eca9-414a-8184-c4daac916e30 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:21.601270 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326588fa-211b-4720-88c4-0bc7e0f7cb5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:21.601912 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326588fa-211b-4720-88c4-0bc7e0f7cb5b admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:21.602187 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326588fa-211b-4720-88c4-0bc7e0f7cb5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:21.602456 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326588fa-211b-4720-88c4-0bc7e0f7cb5b admin admin] PATH_INFO: `/v3/projects/b5d0cf164399493e8ac5c0c854478a74/users/9a8b658af1ab4ce8a169183a61a5f8d9/roles/486144ad1bcd417e86acdbc1d14327bf` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:21.605499 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326588fa-211b-4720-88c4-0bc7e0f7cb5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5d0cf164399493e8ac5c0c854478a74, user_id=9a8b658af1ab4ce8a169183a61a5f8d9, role_id=486144ad1bcd417e86acdbc1d14327bf)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:21.607733 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326588fa-211b-4720-88c4-0bc7e0f7cb5b admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:21.618638 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1339/2697] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:53:21 2022] PUT /identity/v3/projects/b5d0cf164399493e8ac5c0c854478a74/users/9a8b658af1ab4ce8a169183a61a5f8d9/roles/486144ad1bcd417e86acdbc1d14327bf => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:53:21.631311 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-f03924c5-5906-4bcd-8b95-728d5aefe5a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52011aae6d0f483287d4fa527bbc26f3'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:21.631311 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-f03924c5-5906-4bcd-8b95-728d5aefe5a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52011aae6d0f483287d4fa527bbc26f3'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:21.631311 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-f03924c5-5906-4bcd-8b95-728d5aefe5a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52011aae6d0f483287d4fa527bbc26f3'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:21.696148 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-f03924c5-5906-4bcd-8b95-728d5aefe5a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52011aae6d0f483287d4fa527bbc26f3'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:21.709822 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-f03924c5-5906-4bcd-8b95-728d5aefe5a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52011aae6d0f483287d4fa527bbc26f3'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:21.724249 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1359/2698] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:53:21 2022] DELETE /identity/v3/users/52011aae6d0f483287d4fa527bbc26f3 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:53:22.224337 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326588fa-211b-4720-88c4-0bc7e0f7cb5b admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:22.226479 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-326588fa-211b-4720-88c4-0bc7e0f7cb5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:22.303602 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326588fa-211b-4720-88c4-0bc7e0f7cb5b admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:22.319939 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:53:22.379180 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c018cccf-d7a4-4f0a-bdab-46ebd3487a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:22.379879 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c018cccf-d7a4-4f0a-bdab-46ebd3487a6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:22.380163 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c018cccf-d7a4-4f0a-bdab-46ebd3487a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:22.380441 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c018cccf-d7a4-4f0a-bdab-46ebd3487a6d admin admin] PATH_INFO: `/v3/projects/a46d605da0744f3d94ea9f2a42358182` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:22.381667 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c018cccf-d7a4-4f0a-bdab-46ebd3487a6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a46d605da0744f3d94ea9f2a42358182)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:22.383600 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c018cccf-d7a4-4f0a-bdab-46ebd3487a6d admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:22.488601 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-c018cccf-d7a4-4f0a-bdab-46ebd3487a6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a46d605da0744f3d94ea9f2a42358182', 'request_id': 'req-c018cccf-d7a4-4f0a-bdab-46ebd3487a6d'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:22.495799 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-c018cccf-d7a4-4f0a-bdab-46ebd3487a6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a46d605da0744f3d94ea9f2a42358182', 'request_id': 'req-c018cccf-d7a4-4f0a-bdab-46ebd3487a6d'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:22.564477 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-c018cccf-d7a4-4f0a-bdab-46ebd3487a6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a46d605da0744f3d94ea9f2a42358182', 'request_id': 'req-c018cccf-d7a4-4f0a-bdab-46ebd3487a6d'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:22.576907 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-c018cccf-d7a4-4f0a-bdab-46ebd3487a6d admin admin] The token cache is being invalidate because project a46d605da0744f3d94ea9f2a42358182 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:53:22.577368 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-c018cccf-d7a4-4f0a-bdab-46ebd3487a6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:22.578018 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-c018cccf-d7a4-4f0a-bdab-46ebd3487a6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:22.579503 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1340/2699] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:53:22 2022] DELETE /identity/v3/projects/a46d605da0744f3d94ea9f2a42358182 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:53:22.584352 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a80a151-d518-4ef5-81f6-b5eabc2cfdde None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:22.584849 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a80a151-d518-4ef5-81f6-b5eabc2cfdde None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:22.584974 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a80a151-d518-4ef5-81f6-b5eabc2cfdde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:22.606588 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-5a80a151-d518-4ef5-81f6-b5eabc2cfdde None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:53:22.610985 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-5a80a151-d518-4ef5-81f6-b5eabc2cfdde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:22.657555 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:53:22.670274 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1360/2700] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:53:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:53:22.679755 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3b31f2-6a93-47a1-a601-845b7dcadd06 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:22.680283 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3b31f2-6a93-47a1-a601-845b7dcadd06 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:22.680572 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3b31f2-6a93-47a1-a601-845b7dcadd06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:22.702489 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-ee3b31f2-6a93-47a1-a601-845b7dcadd06 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:53:22.703720 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-ee3b31f2-6a93-47a1-a601-845b7dcadd06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:22.718555 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1341/2701] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:53:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:53:22.719802 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a80a151-d518-4ef5-81f6-b5eabc2cfdde None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:22.737683 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a80a151-d518-4ef5-81f6-b5eabc2cfdde None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:22.742245 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7347328-ee52-440c-b35d-15bc28607361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:22.742848 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7347328-ee52-440c-b35d-15bc28607361 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:22.743121 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7347328-ee52-440c-b35d-15bc28607361 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:22.743387 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7347328-ee52-440c-b35d-15bc28607361 admin admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:22.744135 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7347328-ee52-440c-b35d-15bc28607361 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:53:22.744324 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7347328-ee52-440c-b35d-15bc28607361 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:22.746858 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7347328-ee52-440c-b35d-15bc28607361 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:22.760853 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1361/2702] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 24 06:53:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:53:22.766027 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3b31f2-6a93-47a1-a601-845b7dcadd06 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:22.775274 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3b31f2-6a93-47a1-a601-845b7dcadd06 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:22.780775 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df3cf83-538d-4e41-a809-d37b77e29836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:22.781466 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df3cf83-538d-4e41-a809-d37b77e29836 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:22.781757 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df3cf83-538d-4e41-a809-d37b77e29836 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:22.782032 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df3cf83-538d-4e41-a809-d37b77e29836 admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:22.782678 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df3cf83-538d-4e41-a809-d37b77e29836 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:22.784592 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df3cf83-538d-4e41-a809-d37b77e29836 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:22.915503 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1342/2703] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:53:22 2022] POST /identity/v3/projects => generated 387 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:53:22.920639 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7347328-ee52-440c-b35d-15bc28607361 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:22.931014 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7347328-ee52-440c-b35d-15bc28607361 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:22.997360 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600cc96a-a8e3-4146-8ccd-136651e730b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:22.997889 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600cc96a-a8e3-4146-8ccd-136651e730b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:22.998173 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600cc96a-a8e3-4146-8ccd-136651e730b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:22.998444 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600cc96a-a8e3-4146-8ccd-136651e730b1 admin admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:22.999126 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600cc96a-a8e3-4146-8ccd-136651e730b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:23.001258 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600cc96a-a8e3-4146-8ccd-136651e730b1 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:23.035242 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45d30a5-007b-4e1f-94ca-250ce4b0a548 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:23.035522 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45d30a5-007b-4e1f-94ca-250ce4b0a548 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:23.035798 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45d30a5-007b-4e1f-94ca-250ce4b0a548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:23.066557 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-e45d30a5-007b-4e1f-94ca-250ce4b0a548 None None] MFA Rules not processed for user `9a8b658af1ab4ce8a169183a61a5f8d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:53:23.067980 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-e45d30a5-007b-4e1f-94ca-250ce4b0a548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:23.080902 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1362/2704] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:53:22 2022] POST /identity/v3/users => generated 294 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:53:23.085206 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600cc96a-a8e3-4146-8ccd-136651e730b1 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:23.098597 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600cc96a-a8e3-4146-8ccd-136651e730b1 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:23.104290 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebdb00a-1801-46d7-8bbd-e5f2a3abd79f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:23.105173 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebdb00a-1801-46d7-8bbd-e5f2a3abd79f admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:23.105655 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebdb00a-1801-46d7-8bbd-e5f2a3abd79f admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:23.106008 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebdb00a-1801-46d7-8bbd-e5f2a3abd79f admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:23.106738 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebdb00a-1801-46d7-8bbd-e5f2a3abd79f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:23.108440 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebdb00a-1801-46d7-8bbd-e5f2a3abd79f admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:23.124861 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1363/2705] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:53:23 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:53:23.127137 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebdb00a-1801-46d7-8bbd-e5f2a3abd79f admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:23.138832 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebdb00a-1801-46d7-8bbd-e5f2a3abd79f admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:23.143979 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43933c6f-675e-4b49-a216-96de4889052a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:23.144486 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:53:23.144968 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43933c6f-675e-4b49-a216-96de4889052a admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:23.145273 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43933c6f-675e-4b49-a216-96de4889052a admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:23.145545 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43933c6f-675e-4b49-a216-96de4889052a admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:23.146259 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43933c6f-675e-4b49-a216-96de4889052a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:23.148081 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43933c6f-675e-4b49-a216-96de4889052a admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:23.149293 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1343/2706] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:53:23 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:53:23.165792 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1364/2707] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:53:23 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:53:23.170045 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45d30a5-007b-4e1f-94ca-250ce4b0a548 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:23.183442 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45d30a5-007b-4e1f-94ca-250ce4b0a548 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:23.189557 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7aa327-d875-408e-86f2-4a9f948e0fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:23.190265 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7aa327-d875-408e-86f2-4a9f948e0fb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:23.191235 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7aa327-d875-408e-86f2-4a9f948e0fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:23.191235 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7aa327-d875-408e-86f2-4a9f948e0fb8 admin admin] PATH_INFO: `/v3/projects/c2e74cee584d4721a4f20119c24737d8/users/0587e26bea2d4c6abc16a45cdc73955e/roles/e1f9b167c51c4a388cbc9594db8a465d` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:23.207452 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7aa327-d875-408e-86f2-4a9f948e0fb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2e74cee584d4721a4f20119c24737d8, user_id=0587e26bea2d4c6abc16a45cdc73955e, role_id=e1f9b167c51c4a388cbc9594db8a465d)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:23.210280 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7aa327-d875-408e-86f2-4a9f948e0fb8 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:23.326326 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1344/2708] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:53:23 2022] PUT /identity/v3/projects/c2e74cee584d4721a4f20119c24737d8/users/0587e26bea2d4c6abc16a45cdc73955e/roles/e1f9b167c51c4a388cbc9594db8a465d => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:53:23.337331 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bc416f-4a01-43f3-854b-72e43eb15c14 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:23.337611 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bc416f-4a01-43f3-854b-72e43eb15c14 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:23.338142 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bc416f-4a01-43f3-854b-72e43eb15c14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:23.366532 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-64bc416f-4a01-43f3-854b-72e43eb15c14 None None] MFA Rules not processed for user `0587e26bea2d4c6abc16a45cdc73955e`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:53:23.370083 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-64bc416f-4a01-43f3-854b-72e43eb15c14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:23.440629 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1365/2709] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:53:23 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:53:23.445288 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7aa327-d875-408e-86f2-4a9f948e0fb8 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:23.523509 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7aa327-d875-408e-86f2-4a9f948e0fb8 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:23.528479 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9065c9fc-19b5-43ae-9f27-072703ccc86a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:23.529146 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9065c9fc-19b5-43ae-9f27-072703ccc86a admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:23.529455 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9065c9fc-19b5-43ae-9f27-072703ccc86a admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:23.530261 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9065c9fc-19b5-43ae-9f27-072703ccc86a admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:23.530988 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9065c9fc-19b5-43ae-9f27-072703ccc86a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:23.533159 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9065c9fc-19b5-43ae-9f27-072703ccc86a admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:23.631346 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1345/2710] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:53:23 2022] POST /identity/v3/projects => generated 387 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:53:23.636311 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bc416f-4a01-43f3-854b-72e43eb15c14 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:23.647657 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bc416f-4a01-43f3-854b-72e43eb15c14 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:23.689075 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:53:23.708969 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecf226d-5404-42f3-82d0-606b748d0d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:23.709881 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecf226d-5404-42f3-82d0-606b748d0d1e admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:23.710139 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9065c9fc-19b5-43ae-9f27-072703ccc86a admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:23.710325 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecf226d-5404-42f3-82d0-606b748d0d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:23.710677 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecf226d-5404-42f3-82d0-606b748d0d1e admin admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:23.711495 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecf226d-5404-42f3-82d0-606b748d0d1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:23.712072 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-9065c9fc-19b5-43ae-9f27-072703ccc86a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:23.714041 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecf226d-5404-42f3-82d0-606b748d0d1e admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:23.788032 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9065c9fc-19b5-43ae-9f27-072703ccc86a admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:23.793798 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc3a74c-8ec9-47c0-93f0-6ebdac1dead7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '306188d42dbd4b5b84082ab3842eaf9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:23.794411 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc3a74c-8ec9-47c0-93f0-6ebdac1dead7 service placement] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:23.794689 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc3a74c-8ec9-47c0-93f0-6ebdac1dead7 service placement] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:23.794958 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc3a74c-8ec9-47c0-93f0-6ebdac1dead7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:23.796756 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-bcc3a74c-8ec9-47c0-93f0-6ebdac1dead7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:23.810777 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc3a74c-8ec9-47c0-93f0-6ebdac1dead7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:23.811297 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc3a74c-8ec9-47c0-93f0-6ebdac1dead7 service placement] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:23.821894 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1346/2711] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:53:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:53:23.844509 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc3a74c-8ec9-47c0-93f0-6ebdac1dead7 service placement] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:23.878303 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1366/2712] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:53:23 2022] POST /identity/v3/users => generated 300 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:53:23.879413 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc3a74c-8ec9-47c0-93f0-6ebdac1dead7 service placement] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:23.883433 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2abdd1-3709-48ba-ad64-55d1c4ccb292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:23.884285 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2abdd1-3709-48ba-ad64-55d1c4ccb292 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:23.884823 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2abdd1-3709-48ba-ad64-55d1c4ccb292 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:23.885190 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2abdd1-3709-48ba-ad64-55d1c4ccb292 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:23.886756 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2abdd1-3709-48ba-ad64-55d1c4ccb292 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:23.888865 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2abdd1-3709-48ba-ad64-55d1c4ccb292 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:23.899529 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1347/2713] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:53:23 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:53:23.904229 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecf226d-5404-42f3-82d0-606b748d0d1e admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:23.914671 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecf226d-5404-42f3-82d0-606b748d0d1e admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:23.920686 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c800f50c-5031-47c9-9c4a-0dd4fb50a9ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:23.921389 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c800f50c-5031-47c9-9c4a-0dd4fb50a9ff admin admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:23.921672 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c800f50c-5031-47c9-9c4a-0dd4fb50a9ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:23.921946 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c800f50c-5031-47c9-9c4a-0dd4fb50a9ff admin admin] PATH_INFO: `/v3/projects/157d253202bc48ca85f868102c97258a/users/ce7222afb42c4a9e8c0560da7960d9e6/roles/486144ad1bcd417e86acdbc1d14327bf` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:23.939344 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c800f50c-5031-47c9-9c4a-0dd4fb50a9ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=157d253202bc48ca85f868102c97258a, user_id=ce7222afb42c4a9e8c0560da7960d9e6, role_id=486144ad1bcd417e86acdbc1d14327bf)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:23.942045 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c800f50c-5031-47c9-9c4a-0dd4fb50a9ff admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:24.048520 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1367/2714] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:53:23 2022] PUT /identity/v3/projects/157d253202bc48ca85f868102c97258a/users/ce7222afb42c4a9e8c0560da7960d9e6/roles/486144ad1bcd417e86acdbc1d14327bf => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:53:24.060119 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2abdd1-3709-48ba-ad64-55d1c4ccb292 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:24.074117 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2abdd1-3709-48ba-ad64-55d1c4ccb292 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:24.118413 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:53:24.140088 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb530a4-5799-419f-a82f-487b2b13c72e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:24.140708 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb530a4-5799-419f-a82f-487b2b13c72e admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:24.140990 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb530a4-5799-419f-a82f-487b2b13c72e admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:24.141299 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb530a4-5799-419f-a82f-487b2b13c72e admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:24.141941 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb530a4-5799-419f-a82f-487b2b13c72e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:24.143559 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb530a4-5799-419f-a82f-487b2b13c72e admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:24.155053 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1348/2715] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:53:24 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:53:24.159744 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c800f50c-5031-47c9-9c4a-0dd4fb50a9ff admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:24.172555 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c800f50c-5031-47c9-9c4a-0dd4fb50a9ff admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:24.177749 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb09796d-0074-49e7-ae9e-dbc0147e00f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:24.178410 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb09796d-0074-49e7-ae9e-dbc0147e00f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:24.178691 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb09796d-0074-49e7-ae9e-dbc0147e00f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:24.178963 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb09796d-0074-49e7-ae9e-dbc0147e00f0 admin admin] PATH_INFO: `/v3/projects/157d253202bc48ca85f868102c97258a/users/ce7222afb42c4a9e8c0560da7960d9e6/roles/486144ad1bcd417e86acdbc1d14327bf` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:24.181997 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb09796d-0074-49e7-ae9e-dbc0147e00f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=157d253202bc48ca85f868102c97258a, user_id=ce7222afb42c4a9e8c0560da7960d9e6, role_id=486144ad1bcd417e86acdbc1d14327bf)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:24.185987 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb09796d-0074-49e7-ae9e-dbc0147e00f0 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:24.200526 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1368/2716] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:53:24 2022] PUT /identity/v3/projects/157d253202bc48ca85f868102c97258a/users/ce7222afb42c4a9e8c0560da7960d9e6/roles/486144ad1bcd417e86acdbc1d14327bf => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:53:24.220142 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2fa7ef-491f-4a49-99fc-31d00142ce82 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:24.220660 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2fa7ef-491f-4a49-99fc-31d00142ce82 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:24.221155 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2fa7ef-491f-4a49-99fc-31d00142ce82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:24.259127 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-dc2fa7ef-491f-4a49-99fc-31d00142ce82 None None] MFA Rules not processed for user `ce7222afb42c4a9e8c0560da7960d9e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:53:24.260598 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-dc2fa7ef-491f-4a49-99fc-31d00142ce82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:24.336098 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1349/2717] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:53:24 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:53:24.348418 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb09796d-0074-49e7-ae9e-dbc0147e00f0 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:24.350302 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-eb09796d-0074-49e7-ae9e-dbc0147e00f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:24.411724 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb09796d-0074-49e7-ae9e-dbc0147e00f0 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:24.415550 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f89080-9d31-4d3e-a6f4-7f6a1a6feb2e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36639dce4f97412f89d71367f650b901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:24.416228 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f89080-9d31-4d3e-a6f4-7f6a1a6feb2e service cinder] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:24.417489 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f89080-9d31-4d3e-a6f4-7f6a1a6feb2e service cinder] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:24.417925 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f89080-9d31-4d3e-a6f4-7f6a1a6feb2e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:24.427843 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f89080-9d31-4d3e-a6f4-7f6a1a6feb2e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:24.430035 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f89080-9d31-4d3e-a6f4-7f6a1a6feb2e service cinder] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:24.442589 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1369/2718] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:53:24 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:53:24.474895 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2fa7ef-491f-4a49-99fc-31d00142ce82 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:24.549003 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2fa7ef-491f-4a49-99fc-31d00142ce82 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:24.553920 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811bc8f7-8f1b-4a95-8424-b5fe25beedc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:24.554622 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811bc8f7-8f1b-4a95-8424-b5fe25beedc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:24.554887 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811bc8f7-8f1b-4a95-8424-b5fe25beedc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:24.555164 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811bc8f7-8f1b-4a95-8424-b5fe25beedc6 admin admin] PATH_INFO: `/v3/users/0587e26bea2d4c6abc16a45cdc73955e` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:24.556735 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811bc8f7-8f1b-4a95-8424-b5fe25beedc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0587e26bea2d4c6abc16a45cdc73955e)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:24.559272 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811bc8f7-8f1b-4a95-8424-b5fe25beedc6 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:24.603194 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f89080-9d31-4d3e-a6f4-7f6a1a6feb2e service cinder] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:24.603194 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-a0f89080-9d31-4d3e-a6f4-7f6a1a6feb2e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:24.623980 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f89080-9d31-4d3e-a6f4-7f6a1a6feb2e service cinder] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:24.634771 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a1eddb-c904-4557-b017-871593f2a861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:24.635638 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a1eddb-c904-4557-b017-871593f2a861 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:24.635917 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a1eddb-c904-4557-b017-871593f2a861 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:24.636312 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a1eddb-c904-4557-b017-871593f2a861 admin admin] PATH_INFO: `/v3/users/b98979779f414c4e82f24d030ca98197` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:24.639387 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a1eddb-c904-4557-b017-871593f2a861 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b98979779f414c4e82f24d030ca98197)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:24.639387 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a1eddb-c904-4557-b017-871593f2a861 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:24.837484 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-811bc8f7-8f1b-4a95-8424-b5fe25beedc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0587e26bea2d4c6abc16a45cdc73955e'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:24.838401 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-811bc8f7-8f1b-4a95-8424-b5fe25beedc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0587e26bea2d4c6abc16a45cdc73955e'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:24.839183 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-811bc8f7-8f1b-4a95-8424-b5fe25beedc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0587e26bea2d4c6abc16a45cdc73955e'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:24.847335 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-811bc8f7-8f1b-4a95-8424-b5fe25beedc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0587e26bea2d4c6abc16a45cdc73955e'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:24.868658 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-35a1eddb-c904-4557-b017-871593f2a861 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b98979779f414c4e82f24d030ca98197'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:24.871612 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-35a1eddb-c904-4557-b017-871593f2a861 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b98979779f414c4e82f24d030ca98197'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:24.872411 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-35a1eddb-c904-4557-b017-871593f2a861 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b98979779f414c4e82f24d030ca98197'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:24.936249 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-811bc8f7-8f1b-4a95-8424-b5fe25beedc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0587e26bea2d4c6abc16a45cdc73955e'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:24.938202 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:53:24.959057 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1350/2719] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:53:24 2022] DELETE /identity/v3/users/0587e26bea2d4c6abc16a45cdc73955e => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:53:24.967086 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53ca9f3-d8ec-4f5c-8fc8-d11edbbd5749 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:24.967383 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53ca9f3-d8ec-4f5c-8fc8-d11edbbd5749 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:24.967723 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53ca9f3-d8ec-4f5c-8fc8-d11edbbd5749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:24.994058 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-b53ca9f3-d8ec-4f5c-8fc8-d11edbbd5749 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:53:24.996011 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-b53ca9f3-d8ec-4f5c-8fc8-d11edbbd5749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:25.021010 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-35a1eddb-c904-4557-b017-871593f2a861 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b98979779f414c4e82f24d030ca98197'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:25.039099 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-35a1eddb-c904-4557-b017-871593f2a861 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b98979779f414c4e82f24d030ca98197'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:25.059396 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1370/2720] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:53:24 2022] DELETE /identity/v3/users/b98979779f414c4e82f24d030ca98197 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:53:25.074257 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1351/2721] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:53:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:53:25.085494 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a1eddb-c904-4557-b017-871593f2a861 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:25.086982 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-35a1eddb-c904-4557-b017-871593f2a861 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:25.102754 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:53:25.171789 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a1eddb-c904-4557-b017-871593f2a861 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:25.175520 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50216cbd-7e01-44f8-aada-e321a2a40c5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:25.176133 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50216cbd-7e01-44f8-aada-e321a2a40c5e service neutron] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:25.176414 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50216cbd-7e01-44f8-aada-e321a2a40c5e service neutron] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:25.176757 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50216cbd-7e01-44f8-aada-e321a2a40c5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:25.250067 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50216cbd-7e01-44f8-aada-e321a2a40c5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:25.251698 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50216cbd-7e01-44f8-aada-e321a2a40c5e service neutron] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:25.271623 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1371/2722] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:53:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:53:25.442476 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53ca9f3-d8ec-4f5c-8fc8-d11edbbd5749 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:25.508129 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53ca9f3-d8ec-4f5c-8fc8-d11edbbd5749 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:25.512499 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76288f57-ea39-47ad-9359-0111a6102775 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:25.513380 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76288f57-ea39-47ad-9359-0111a6102775 service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:25.513762 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76288f57-ea39-47ad-9359-0111a6102775 service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:25.514247 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76288f57-ea39-47ad-9359-0111a6102775 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:25.519929 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-76288f57-ea39-47ad-9359-0111a6102775 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:25.525138 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50216cbd-7e01-44f8-aada-e321a2a40c5e service neutron] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:25.525138 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-50216cbd-7e01-44f8-aada-e321a2a40c5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:25.528635 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76288f57-ea39-47ad-9359-0111a6102775 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:25.530630 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76288f57-ea39-47ad-9359-0111a6102775 service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:25.540745 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1352/2723] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:53:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:53:25.559704 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50216cbd-7e01-44f8-aada-e321a2a40c5e service neutron] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:25.565588 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671fd63d-8bea-452d-a210-907cfc083cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:25.567114 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671fd63d-8bea-452d-a210-907cfc083cf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:25.567701 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671fd63d-8bea-452d-a210-907cfc083cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:25.568201 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671fd63d-8bea-452d-a210-907cfc083cf8 admin admin] PATH_INFO: `/v3/projects/153fb2e409044a1f8aa05c7cdb3979db` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:25.570588 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671fd63d-8bea-452d-a210-907cfc083cf8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=153fb2e409044a1f8aa05c7cdb3979db)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:25.573044 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671fd63d-8bea-452d-a210-907cfc083cf8 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:25.722070 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-671fd63d-8bea-452d-a210-907cfc083cf8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '153fb2e409044a1f8aa05c7cdb3979db', 'request_id': 'req-671fd63d-8bea-452d-a210-907cfc083cf8'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:25.848553 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-671fd63d-8bea-452d-a210-907cfc083cf8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '153fb2e409044a1f8aa05c7cdb3979db', 'request_id': 'req-671fd63d-8bea-452d-a210-907cfc083cf8'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:25.859827 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-671fd63d-8bea-452d-a210-907cfc083cf8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '153fb2e409044a1f8aa05c7cdb3979db', 'request_id': 'req-671fd63d-8bea-452d-a210-907cfc083cf8'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:25.872871 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-671fd63d-8bea-452d-a210-907cfc083cf8 admin admin] The token cache is being invalidate because project 153fb2e409044a1f8aa05c7cdb3979db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81723) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:53:25.873421 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-671fd63d-8bea-452d-a210-907cfc083cf8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:25.874335 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-671fd63d-8bea-452d-a210-907cfc083cf8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:25.875903 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1372/2724] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:53:25 2022] DELETE /identity/v3/projects/153fb2e409044a1f8aa05c7cdb3979db => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:53:25.883650 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76288f57-ea39-47ad-9359-0111a6102775 service neutron] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:25.887511 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671fd63d-8bea-452d-a210-907cfc083cf8 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:25.888919 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-671fd63d-8bea-452d-a210-907cfc083cf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:25.895413 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76288f57-ea39-47ad-9359-0111a6102775 service neutron] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:25.982080 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:53:26.007508 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b5a5b4-d2de-423b-bad4-9412da807aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:26.008320 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b5a5b4-d2de-423b-bad4-9412da807aa8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:26.008859 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b5a5b4-d2de-423b-bad4-9412da807aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:26.009208 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671fd63d-8bea-452d-a210-907cfc083cf8 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:26.009341 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b5a5b4-d2de-423b-bad4-9412da807aa8 admin admin] PATH_INFO: `/v3/users/9a8b658af1ab4ce8a169183a61a5f8d9` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:26.011084 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b5a5b4-d2de-423b-bad4-9412da807aa8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a8b658af1ab4ce8a169183a61a5f8d9)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:26.013291 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b5a5b4-d2de-423b-bad4-9412da807aa8 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:26.015202 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406b7644-33b3-4c1e-859b-ce910bf6b347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:26.015839 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b7644-33b3-4c1e-859b-ce910bf6b347 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:26.016122 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b7644-33b3-4c1e-859b-ce910bf6b347 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:26.016393 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b7644-33b3-4c1e-859b-ce910bf6b347 admin admin] PATH_INFO: `/v3/projects/c2e74cee584d4721a4f20119c24737d8` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:26.017584 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406b7644-33b3-4c1e-859b-ce910bf6b347 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2e74cee584d4721a4f20119c24737d8)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:26.019914 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406b7644-33b3-4c1e-859b-ce910bf6b347 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:26.137360 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:53:26.263586 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-406b7644-33b3-4c1e-859b-ce910bf6b347 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2e74cee584d4721a4f20119c24737d8', 'request_id': 'req-406b7644-33b3-4c1e-859b-ce910bf6b347'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:26.270120 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-e3b5a5b4-d2de-423b-bad4-9412da807aa8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a8b658af1ab4ce8a169183a61a5f8d9'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:26.270998 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-e3b5a5b4-d2de-423b-bad4-9412da807aa8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a8b658af1ab4ce8a169183a61a5f8d9'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:26.271693 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-e3b5a5b4-d2de-423b-bad4-9412da807aa8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a8b658af1ab4ce8a169183a61a5f8d9'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:26.280037 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-e3b5a5b4-d2de-423b-bad4-9412da807aa8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a8b658af1ab4ce8a169183a61a5f8d9'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:26.364785 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-406b7644-33b3-4c1e-859b-ce910bf6b347 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2e74cee584d4721a4f20119c24737d8', 'request_id': 'req-406b7644-33b3-4c1e-859b-ce910bf6b347'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:26.378396 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-406b7644-33b3-4c1e-859b-ce910bf6b347 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2e74cee584d4721a4f20119c24737d8', 'request_id': 'req-406b7644-33b3-4c1e-859b-ce910bf6b347'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:26.392682 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-406b7644-33b3-4c1e-859b-ce910bf6b347 admin admin] The token cache is being invalidate because project c2e74cee584d4721a4f20119c24737d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81723) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:53:26.392970 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-406b7644-33b3-4c1e-859b-ce910bf6b347 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:26.393694 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-406b7644-33b3-4c1e-859b-ce910bf6b347 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:26.395553 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1373/2725] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:53:25 2022] DELETE /identity/v3/projects/c2e74cee584d4721a4f20119c24737d8 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:53:26.401636 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406b7644-33b3-4c1e-859b-ce910bf6b347 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:26.403048 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-406b7644-33b3-4c1e-859b-ce910bf6b347 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:26.450027 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-e3b5a5b4-d2de-423b-bad4-9412da807aa8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a8b658af1ab4ce8a169183a61a5f8d9'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:26.470121 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1353/2726] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:53:25 2022] DELETE /identity/v3/users/9a8b658af1ab4ce8a169183a61a5f8d9 => generated 0 bytes in 590 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:53:26.475501 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406b7644-33b3-4c1e-859b-ce910bf6b347 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:26.542702 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65597873-6564-4b77-a6a7-034fd4dfee7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:26.543498 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65597873-6564-4b77-a6a7-034fd4dfee7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:26.543838 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65597873-6564-4b77-a6a7-034fd4dfee7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:26.544179 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65597873-6564-4b77-a6a7-034fd4dfee7d admin admin] PATH_INFO: `/v3/users/ce7222afb42c4a9e8c0560da7960d9e6` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:26.547568 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65597873-6564-4b77-a6a7-034fd4dfee7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce7222afb42c4a9e8c0560da7960d9e6)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:26.549699 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65597873-6564-4b77-a6a7-034fd4dfee7d admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:26.623521 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b5a5b4-d2de-423b-bad4-9412da807aa8 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:26.625103 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-e3b5a5b4-d2de-423b-bad4-9412da807aa8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:26.757777 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-65597873-6564-4b77-a6a7-034fd4dfee7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce7222afb42c4a9e8c0560da7960d9e6'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:26.760616 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-65597873-6564-4b77-a6a7-034fd4dfee7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce7222afb42c4a9e8c0560da7960d9e6'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:26.760616 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-65597873-6564-4b77-a6a7-034fd4dfee7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce7222afb42c4a9e8c0560da7960d9e6'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:26.780863 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b5a5b4-d2de-423b-bad4-9412da807aa8 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:26.792528 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ba9452-b3e9-444f-bd62-eff57ab13f0c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '306188d42dbd4b5b84082ab3842eaf9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:26.794680 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ba9452-b3e9-444f-bd62-eff57ab13f0c service placement] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:26.794680 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ba9452-b3e9-444f-bd62-eff57ab13f0c service placement] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:26.794680 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ba9452-b3e9-444f-bd62-eff57ab13f0c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:26.796495 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-42ba9452-b3e9-444f-bd62-eff57ab13f0c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:26.824676 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ba9452-b3e9-444f-bd62-eff57ab13f0c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:26.826604 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ba9452-b3e9-444f-bd62-eff57ab13f0c service placement] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:26.827760 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-65597873-6564-4b77-a6a7-034fd4dfee7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce7222afb42c4a9e8c0560da7960d9e6'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:26.845441 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-65597873-6564-4b77-a6a7-034fd4dfee7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce7222afb42c4a9e8c0560da7960d9e6'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:26.856158 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1354/2727] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:53:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:53:26.857047 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ba9452-b3e9-444f-bd62-eff57ab13f0c service placement] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:26.864678 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-42ba9452-b3e9-444f-bd62-eff57ab13f0c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:26.885477 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ba9452-b3e9-444f-bd62-eff57ab13f0c service placement] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:26.895206 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f676397-dd62-46f6-9753-aa03ef08fd11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:26.895390 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1374/2728] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:53:26 2022] DELETE /identity/v3/users/ce7222afb42c4a9e8c0560da7960d9e6 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:53:26.895519 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f676397-dd62-46f6-9753-aa03ef08fd11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:26.895519 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f676397-dd62-46f6-9753-aa03ef08fd11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:26.895519 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f676397-dd62-46f6-9753-aa03ef08fd11 admin admin] PATH_INFO: `/v3/projects/b5d0cf164399493e8ac5c0c854478a74` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:26.910015 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f676397-dd62-46f6-9753-aa03ef08fd11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5d0cf164399493e8ac5c0c854478a74)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:26.919300 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f676397-dd62-46f6-9753-aa03ef08fd11 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:27.086819 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-3f676397-dd62-46f6-9753-aa03ef08fd11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5d0cf164399493e8ac5c0c854478a74', 'request_id': 'req-3f676397-dd62-46f6-9753-aa03ef08fd11'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:27.094129 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-3f676397-dd62-46f6-9753-aa03ef08fd11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5d0cf164399493e8ac5c0c854478a74', 'request_id': 'req-3f676397-dd62-46f6-9753-aa03ef08fd11'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:27.130349 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:53:27.181677 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-3f676397-dd62-46f6-9753-aa03ef08fd11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5d0cf164399493e8ac5c0c854478a74', 'request_id': 'req-3f676397-dd62-46f6-9753-aa03ef08fd11'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:27.200250 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-3f676397-dd62-46f6-9753-aa03ef08fd11 admin admin] The token cache is being invalidate because project b5d0cf164399493e8ac5c0c854478a74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:53:27.201018 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-3f676397-dd62-46f6-9753-aa03ef08fd11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:27.201916 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-3f676397-dd62-46f6-9753-aa03ef08fd11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:27.208145 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1355/2729] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:53:26 2022] DELETE /identity/v3/projects/b5d0cf164399493e8ac5c0c854478a74 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:53:27.209839 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440ddc96-df98-4bb3-9fcb-5bac5532ad2e None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:27.212211 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440ddc96-df98-4bb3-9fcb-5bac5532ad2e None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:27.212211 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440ddc96-df98-4bb3-9fcb-5bac5532ad2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:27.236375 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-440ddc96-df98-4bb3-9fcb-5bac5532ad2e None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:53:27.237735 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-440ddc96-df98-4bb3-9fcb-5bac5532ad2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:27.331459 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1375/2730] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:53:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:53:27.334795 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f12fe1c-43f7-4cc4-a662-b0308ae8b248 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:27.334795 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f12fe1c-43f7-4cc4-a662-b0308ae8b248 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:27.335151 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f12fe1c-43f7-4cc4-a662-b0308ae8b248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:27.357722 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-7f12fe1c-43f7-4cc4-a662-b0308ae8b248 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:53:27.358966 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-7f12fe1c-43f7-4cc4-a662-b0308ae8b248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:27.373230 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1356/2731] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:53:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:53:27.376334 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440ddc96-df98-4bb3-9fcb-5bac5532ad2e None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:27.387346 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440ddc96-df98-4bb3-9fcb-5bac5532ad2e None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:27.392339 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7588d6-9082-407d-bc33-4c3f994582f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:27.392958 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7588d6-9082-407d-bc33-4c3f994582f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:27.393287 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7588d6-9082-407d-bc33-4c3f994582f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:27.393564 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7588d6-9082-407d-bc33-4c3f994582f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:27.394244 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7588d6-9082-407d-bc33-4c3f994582f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:53:27.394550 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7588d6-9082-407d-bc33-4c3f994582f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:27.396166 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7588d6-9082-407d-bc33-4c3f994582f2 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:27.411063 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1376/2732] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 24 06:53:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:53:27.415716 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f12fe1c-43f7-4cc4-a662-b0308ae8b248 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:27.426465 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f12fe1c-43f7-4cc4-a662-b0308ae8b248 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:27.431413 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecccc54-430d-4522-9ca9-1b84405fec39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:27.432641 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecccc54-430d-4522-9ca9-1b84405fec39 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:27.432957 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecccc54-430d-4522-9ca9-1b84405fec39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:27.433388 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecccc54-430d-4522-9ca9-1b84405fec39 admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:27.434488 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecccc54-430d-4522-9ca9-1b84405fec39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:27.436290 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecccc54-430d-4522-9ca9-1b84405fec39 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:27.442804 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:53:27.459439 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7588d6-9082-407d-bc33-4c3f994582f2 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:27.461961 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-7e7588d6-9082-407d-bc33-4c3f994582f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:27.497062 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7588d6-9082-407d-bc33-4c3f994582f2 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:27.502026 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06cd6a3-f6a7-4218-b430-62c95b2fbae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:27.506167 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06cd6a3-f6a7-4218-b430-62c95b2fbae3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:27.506167 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06cd6a3-f6a7-4218-b430-62c95b2fbae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:27.506167 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06cd6a3-f6a7-4218-b430-62c95b2fbae3 admin admin] PATH_INFO: `/v3/projects/157d253202bc48ca85f868102c97258a` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:27.506167 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06cd6a3-f6a7-4218-b430-62c95b2fbae3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=157d253202bc48ca85f868102c97258a)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:27.506545 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06cd6a3-f6a7-4218-b430-62c95b2fbae3 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:27.572839 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1357/2733] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:53:27 2022] POST /identity/v3/projects => generated 373 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:53:27.577390 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecccc54-430d-4522-9ca9-1b84405fec39 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:27.650260 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecccc54-430d-4522-9ca9-1b84405fec39 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:27.656459 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673abab4-5d82-4b4c-8533-b0c005f2a356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:27.657092 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673abab4-5d82-4b4c-8533-b0c005f2a356 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:27.657401 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673abab4-5d82-4b4c-8533-b0c005f2a356 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:27.657765 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673abab4-5d82-4b4c-8533-b0c005f2a356 admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:27.658368 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673abab4-5d82-4b4c-8533-b0c005f2a356 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:27.660452 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673abab4-5d82-4b4c-8533-b0c005f2a356 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:27.695611 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-b06cd6a3-f6a7-4218-b430-62c95b2fbae3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '157d253202bc48ca85f868102c97258a', 'request_id': 'req-b06cd6a3-f6a7-4218-b430-62c95b2fbae3'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:27.749959 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1358/2734] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:53:27 2022] POST /identity/v3/users => generated 287 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:53:27.754124 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673abab4-5d82-4b4c-8533-b0c005f2a356 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:27.764569 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-b06cd6a3-f6a7-4218-b430-62c95b2fbae3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '157d253202bc48ca85f868102c97258a', 'request_id': 'req-b06cd6a3-f6a7-4218-b430-62c95b2fbae3'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:27.767453 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673abab4-5d82-4b4c-8533-b0c005f2a356 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:27.772610 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-b06cd6a3-f6a7-4218-b430-62c95b2fbae3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '157d253202bc48ca85f868102c97258a', 'request_id': 'req-b06cd6a3-f6a7-4218-b430-62c95b2fbae3'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:27.785333 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-b06cd6a3-f6a7-4218-b430-62c95b2fbae3 admin admin] The token cache is being invalidate because project 157d253202bc48ca85f868102c97258a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81723) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:53:27.786083 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-b06cd6a3-f6a7-4218-b430-62c95b2fbae3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:27.787086 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-b06cd6a3-f6a7-4218-b430-62c95b2fbae3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:27.789167 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1377/2735] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:53:27 2022] DELETE /identity/v3/projects/157d253202bc48ca85f868102c97258a => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:53:27.794918 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadf3df1-6cab-41b2-a811-53ac7f95dfb7 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:27.795206 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadf3df1-6cab-41b2-a811-53ac7f95dfb7 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:27.795483 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadf3df1-6cab-41b2-a811-53ac7f95dfb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:27.813602 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:53:27.829858 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-eadf3df1-6cab-41b2-a811-53ac7f95dfb7 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:53:27.830715 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-eadf3df1-6cab-41b2-a811-53ac7f95dfb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:27.853743 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277aff60-b01d-41ab-af49-6e9e2e641633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:27.855232 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277aff60-b01d-41ab-af49-6e9e2e641633 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:27.855528 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277aff60-b01d-41ab-af49-6e9e2e641633 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:27.855944 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277aff60-b01d-41ab-af49-6e9e2e641633 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:27.856768 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277aff60-b01d-41ab-af49-6e9e2e641633 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:27.858446 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277aff60-b01d-41ab-af49-6e9e2e641633 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:27.871794 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1359/2736] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:53:27 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:53:27.875700 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277aff60-b01d-41ab-af49-6e9e2e641633 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:27.877586 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-277aff60-b01d-41ab-af49-6e9e2e641633 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:27.892570 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277aff60-b01d-41ab-af49-6e9e2e641633 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:27.901774 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96b7e3b-6d47-495e-a98a-74cbfa01769a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:27.903771 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96b7e3b-6d47-495e-a98a-74cbfa01769a admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:27.903771 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96b7e3b-6d47-495e-a98a-74cbfa01769a admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:27.903771 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96b7e3b-6d47-495e-a98a-74cbfa01769a admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:27.903771 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96b7e3b-6d47-495e-a98a-74cbfa01769a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:27.905338 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96b7e3b-6d47-495e-a98a-74cbfa01769a admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:27.921374 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1360/2737] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:53:27 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:53:27.921841 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1378/2738] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:53:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:53:27.923591 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96b7e3b-6d47-495e-a98a-74cbfa01769a admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:27.930271 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9d054a-ffa8-4f64-965e-ad80444a3de3 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:27.932337 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9d054a-ffa8-4f64-965e-ad80444a3de3 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:27.932337 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9d054a-ffa8-4f64-965e-ad80444a3de3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:27.935500 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96b7e3b-6d47-495e-a98a-74cbfa01769a admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:27.945838 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5688f5e1-3ed1-4227-8320-dcbdb152c2ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:27.947655 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5688f5e1-3ed1-4227-8320-dcbdb152c2ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:27.947973 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5688f5e1-3ed1-4227-8320-dcbdb152c2ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:27.948498 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5688f5e1-3ed1-4227-8320-dcbdb152c2ab admin admin] PATH_INFO: `/v3/projects/0f8d0d85723a4832897bebcdb48327bd/users/d1996f7e94a749cdbd9bcfdfb6a61f04/roles/e1f9b167c51c4a388cbc9594db8a465d` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:27.955461 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-4c9d054a-ffa8-4f64-965e-ad80444a3de3 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:53:27.955671 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-4c9d054a-ffa8-4f64-965e-ad80444a3de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:27.964909 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1379/2739] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:53:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:53:27.966890 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5688f5e1-3ed1-4227-8320-dcbdb152c2ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f8d0d85723a4832897bebcdb48327bd, user_id=d1996f7e94a749cdbd9bcfdfb6a61f04, role_id=e1f9b167c51c4a388cbc9594db8a465d)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:27.969825 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5688f5e1-3ed1-4227-8320-dcbdb152c2ab admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:27.970165 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9d054a-ffa8-4f64-965e-ad80444a3de3 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:27.983083 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9d054a-ffa8-4f64-965e-ad80444a3de3 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:27.988971 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911988fb-f45f-4aeb-a41c-65156678b1f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:27.989798 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911988fb-f45f-4aeb-a41c-65156678b1f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:27.990184 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911988fb-f45f-4aeb-a41c-65156678b1f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:27.990652 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911988fb-f45f-4aeb-a41c-65156678b1f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:27.991810 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911988fb-f45f-4aeb-a41c-65156678b1f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:53:27.992288 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911988fb-f45f-4aeb-a41c-65156678b1f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:27.994409 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911988fb-f45f-4aeb-a41c-65156678b1f6 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:28.007004 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1380/2740] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 24 06:53:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:53:28.012490 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911988fb-f45f-4aeb-a41c-65156678b1f6 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:28.022294 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911988fb-f45f-4aeb-a41c-65156678b1f6 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:28.030832 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec98d78-602b-4a31-9aad-93466fe97bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:28.031483 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec98d78-602b-4a31-9aad-93466fe97bc3 admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:28.031798 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec98d78-602b-4a31-9aad-93466fe97bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:28.032124 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec98d78-602b-4a31-9aad-93466fe97bc3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:28.032748 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec98d78-602b-4a31-9aad-93466fe97bc3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:28.034961 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec98d78-602b-4a31-9aad-93466fe97bc3 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:28.059135 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1361/2741] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:53:27 2022] PUT /identity/v3/projects/0f8d0d85723a4832897bebcdb48327bd/users/d1996f7e94a749cdbd9bcfdfb6a61f04/roles/e1f9b167c51c4a388cbc9594db8a465d => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:53:28.073203 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13b13ff-4f40-4f0f-b927-47b8470dd1f7 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:28.073655 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13b13ff-4f40-4f0f-b927-47b8470dd1f7 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:28.074516 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13b13ff-4f40-4f0f-b927-47b8470dd1f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:28.127963 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-d13b13ff-4f40-4f0f-b927-47b8470dd1f7 None None] MFA Rules not processed for user `d1996f7e94a749cdbd9bcfdfb6a61f04`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:53:28.129713 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-d13b13ff-4f40-4f0f-b927-47b8470dd1f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:28.164616 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1381/2742] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:53:28 2022] POST /identity/v3/projects => generated 385 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:53:28.165212 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:53:28.165983 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec98d78-602b-4a31-9aad-93466fe97bc3 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:28.227604 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1362/2743] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:53:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:53:28.232227 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13b13ff-4f40-4f0f-b927-47b8470dd1f7 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:28.240910 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec98d78-602b-4a31-9aad-93466fe97bc3 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:28.246243 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b9feef-baef-45be-b49a-3e2532d89b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:28.246874 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b9feef-baef-45be-b49a-3e2532d89b7d admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:28.247185 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b9feef-baef-45be-b49a-3e2532d89b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:28.247481 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b9feef-baef-45be-b49a-3e2532d89b7d admin admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:28.248136 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b9feef-baef-45be-b49a-3e2532d89b7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:28.250319 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b9feef-baef-45be-b49a-3e2532d89b7d admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:28.299437 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13b13ff-4f40-4f0f-b927-47b8470dd1f7 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:28.304151 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242e8858-cbd5-4909-bc1a-ff4f877d2c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:28.304750 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242e8858-cbd5-4909-bc1a-ff4f877d2c0d admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:28.305026 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242e8858-cbd5-4909-bc1a-ff4f877d2c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:28.305325 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242e8858-cbd5-4909-bc1a-ff4f877d2c0d admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:28.305937 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242e8858-cbd5-4909-bc1a-ff4f877d2c0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:28.308064 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242e8858-cbd5-4909-bc1a-ff4f877d2c0d admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:28.330651 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1382/2744] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:53:28 2022] POST /identity/v3/users => generated 293 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:53:28.336068 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b9feef-baef-45be-b49a-3e2532d89b7d admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:28.347496 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b9feef-baef-45be-b49a-3e2532d89b7d admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:28.352529 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3a1f69-67b5-4a81-b9d0-70002ea92416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:28.353130 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3a1f69-67b5-4a81-b9d0-70002ea92416 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:28.353466 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3a1f69-67b5-4a81-b9d0-70002ea92416 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:28.353854 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3a1f69-67b5-4a81-b9d0-70002ea92416 admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:28.354592 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3a1f69-67b5-4a81-b9d0-70002ea92416 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:28.356189 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3a1f69-67b5-4a81-b9d0-70002ea92416 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:28.366493 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1383/2745] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:53:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:53:28.371721 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3a1f69-67b5-4a81-b9d0-70002ea92416 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:28.381825 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3a1f69-67b5-4a81-b9d0-70002ea92416 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:28.387096 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75849350-f620-4b43-9f3d-5e6fbb319a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:28.395024 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75849350-f620-4b43-9f3d-5e6fbb319a3f admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:28.395024 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75849350-f620-4b43-9f3d-5e6fbb319a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:28.395024 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75849350-f620-4b43-9f3d-5e6fbb319a3f admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:28.395024 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75849350-f620-4b43-9f3d-5e6fbb319a3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:28.395024 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75849350-f620-4b43-9f3d-5e6fbb319a3f admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:28.402417 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1384/2746] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:53:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:53:28.407421 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75849350-f620-4b43-9f3d-5e6fbb319a3f admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:28.418503 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75849350-f620-4b43-9f3d-5e6fbb319a3f admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:28.424142 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9680c1-50de-40e8-827a-610421e76a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:28.424787 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1363/2747] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:53:28 2022] POST /identity/v3/projects => generated 373 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:53:28.424926 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9680c1-50de-40e8-827a-610421e76a84 admin admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:28.425127 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9680c1-50de-40e8-827a-610421e76a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:28.425441 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9680c1-50de-40e8-827a-610421e76a84 admin admin] PATH_INFO: `/v3/projects/98be857d628b42e6b6fe8f2f45b7dd74/users/d08b7dc4929045a0ac649bbbc9f46e3d/roles/e1f9b167c51c4a388cbc9594db8a465d` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:28.429115 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242e8858-cbd5-4909-bc1a-ff4f877d2c0d admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:28.441464 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9680c1-50de-40e8-827a-610421e76a84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98be857d628b42e6b6fe8f2f45b7dd74, user_id=d08b7dc4929045a0ac649bbbc9f46e3d, role_id=e1f9b167c51c4a388cbc9594db8a465d)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:28.444115 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9680c1-50de-40e8-827a-610421e76a84 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:28.498349 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242e8858-cbd5-4909-bc1a-ff4f877d2c0d admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:28.503341 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253e77b8-cd4d-420a-9455-30c6e940902b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:28.503959 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253e77b8-cd4d-420a-9455-30c6e940902b admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:28.504241 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253e77b8-cd4d-420a-9455-30c6e940902b admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:28.504526 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253e77b8-cd4d-420a-9455-30c6e940902b admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:28.505152 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253e77b8-cd4d-420a-9455-30c6e940902b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:28.507270 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253e77b8-cd4d-420a-9455-30c6e940902b admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:28.543932 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1385/2748] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:53:28 2022] PUT /identity/v3/projects/98be857d628b42e6b6fe8f2f45b7dd74/users/d08b7dc4929045a0ac649bbbc9f46e3d/roles/e1f9b167c51c4a388cbc9594db8a465d => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:53:28.554992 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f38ec65-2ed9-49f6-bdb5-62cd30d96cd4 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:28.555290 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f38ec65-2ed9-49f6-bdb5-62cd30d96cd4 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:28.555713 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f38ec65-2ed9-49f6-bdb5-62cd30d96cd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:28.579295 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:53:28.595127 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-0f38ec65-2ed9-49f6-bdb5-62cd30d96cd4 None None] MFA Rules not processed for user `d08b7dc4929045a0ac649bbbc9f46e3d`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:53:28.596417 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-0f38ec65-2ed9-49f6-bdb5-62cd30d96cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:28.617662 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1364/2749] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:53:28 2022] POST /identity/v3/users => generated 293 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:53:28.622392 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253e77b8-cd4d-420a-9455-30c6e940902b admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:28.642718 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253e77b8-cd4d-420a-9455-30c6e940902b admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:28.711926 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1386/2750] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:53:28 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:53:28.716239 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f38ec65-2ed9-49f6-bdb5-62cd30d96cd4 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:28.719449 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:53:28.740715 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffd9b1b-cd2e-458e-abfe-4a7b17e50398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:28.741381 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffd9b1b-cd2e-458e-abfe-4a7b17e50398 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:28.741668 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffd9b1b-cd2e-458e-abfe-4a7b17e50398 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:28.741948 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffd9b1b-cd2e-458e-abfe-4a7b17e50398 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:28.742602 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffd9b1b-cd2e-458e-abfe-4a7b17e50398 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:28.744267 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffd9b1b-cd2e-458e-abfe-4a7b17e50398 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:28.759850 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1365/2751] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:53:28 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:53:28.764667 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffd9b1b-cd2e-458e-abfe-4a7b17e50398 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:28.781395 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffd9b1b-cd2e-458e-abfe-4a7b17e50398 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:28.795862 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb36f0ab-34de-455a-8fc3-2aa6e897954a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:28.803235 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb36f0ab-34de-455a-8fc3-2aa6e897954a admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:28.803235 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb36f0ab-34de-455a-8fc3-2aa6e897954a admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:28.803235 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb36f0ab-34de-455a-8fc3-2aa6e897954a admin admin] PATH_INFO: `/v3/projects/0ba95bda3c6f46a5b49d5c367b21606b/users/8378987fa33b45ccaf9350acb1516857/roles/486144ad1bcd417e86acdbc1d14327bf` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:28.819432 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb36f0ab-34de-455a-8fc3-2aa6e897954a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ba95bda3c6f46a5b49d5c367b21606b, user_id=8378987fa33b45ccaf9350acb1516857, role_id=486144ad1bcd417e86acdbc1d14327bf)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:28.822706 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb36f0ab-34de-455a-8fc3-2aa6e897954a admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:28.834637 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f38ec65-2ed9-49f6-bdb5-62cd30d96cd4 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:28.840987 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a80f22-bb03-4f0c-a37e-4e35fd43ce94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:28.841760 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a80f22-bb03-4f0c-a37e-4e35fd43ce94 admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:28.842077 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a80f22-bb03-4f0c-a37e-4e35fd43ce94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:28.842355 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a80f22-bb03-4f0c-a37e-4e35fd43ce94 admin admin] PATH_INFO: `/v3/projects` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:28.843014 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a80f22-bb03-4f0c-a37e-4e35fd43ce94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:28.845052 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a80f22-bb03-4f0c-a37e-4e35fd43ce94 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:28.948934 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1366/2752] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:53:28 2022] PUT /identity/v3/projects/0ba95bda3c6f46a5b49d5c367b21606b/users/8378987fa33b45ccaf9350acb1516857/roles/486144ad1bcd417e86acdbc1d14327bf => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:53:28.954422 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb36f0ab-34de-455a-8fc3-2aa6e897954a admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:28.980275 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1387/2753] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:53:28 2022] POST /identity/v3/projects => generated 383 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:53:28.984853 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a80f22-bb03-4f0c-a37e-4e35fd43ce94 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:29.066476 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a80f22-bb03-4f0c-a37e-4e35fd43ce94 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:29.067261 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb36f0ab-34de-455a-8fc3-2aa6e897954a admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:29.071994 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7911c6c2-21f5-4c7d-ada4-73aae774cca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:29.073329 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7911c6c2-21f5-4c7d-ada4-73aae774cca4 admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:29.073455 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7911c6c2-21f5-4c7d-ada4-73aae774cca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:29.073640 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126f059d-c83b-42d3-a0e6-3927ec8f6677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:29.078881 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7911c6c2-21f5-4c7d-ada4-73aae774cca4 admin admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:29.079058 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126f059d-c83b-42d3-a0e6-3927ec8f6677 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:29.079148 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7911c6c2-21f5-4c7d-ada4-73aae774cca4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:29.079238 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126f059d-c83b-42d3-a0e6-3927ec8f6677 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:29.079238 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126f059d-c83b-42d3-a0e6-3927ec8f6677 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:29.079238 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126f059d-c83b-42d3-a0e6-3927ec8f6677 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:29.079483 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7911c6c2-21f5-4c7d-ada4-73aae774cca4 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:29.079565 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126f059d-c83b-42d3-a0e6-3927ec8f6677 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:29.093416 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1367/2754] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:53:28 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:53:29.097777 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126f059d-c83b-42d3-a0e6-3927ec8f6677 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:29.108818 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126f059d-c83b-42d3-a0e6-3927ec8f6677 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:29.114065 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e12e92a-6df1-455a-b78f-f3301fae985a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:29.114735 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e12e92a-6df1-455a-b78f-f3301fae985a admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:29.115012 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e12e92a-6df1-455a-b78f-f3301fae985a admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:29.115284 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e12e92a-6df1-455a-b78f-f3301fae985a admin admin] PATH_INFO: `/v3/projects/0ba95bda3c6f46a5b49d5c367b21606b/users/8378987fa33b45ccaf9350acb1516857/roles/486144ad1bcd417e86acdbc1d14327bf` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:29.117372 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e12e92a-6df1-455a-b78f-f3301fae985a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ba95bda3c6f46a5b49d5c367b21606b, user_id=8378987fa33b45ccaf9350acb1516857, role_id=486144ad1bcd417e86acdbc1d14327bf)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:29.120470 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e12e92a-6df1-455a-b78f-f3301fae985a admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:29.131313 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1368/2755] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:53:29 2022] PUT /identity/v3/projects/0ba95bda3c6f46a5b49d5c367b21606b/users/8378987fa33b45ccaf9350acb1516857/roles/486144ad1bcd417e86acdbc1d14327bf => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:53:29.142536 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13aa046-b10c-44ad-bdff-6964439366d7 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:29.142827 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13aa046-b10c-44ad-bdff-6964439366d7 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:29.147747 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13aa046-b10c-44ad-bdff-6964439366d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:29.171787 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1388/2756] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:53:28 2022] POST /identity/v3/users => generated 298 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:53:29.173361 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-c13aa046-b10c-44ad-bdff-6964439366d7 None None] MFA Rules not processed for user `8378987fa33b45ccaf9350acb1516857`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:53:29.174710 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-c13aa046-b10c-44ad-bdff-6964439366d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:29.175426 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7911c6c2-21f5-4c7d-ada4-73aae774cca4 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:29.186001 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7911c6c2-21f5-4c7d-ada4-73aae774cca4 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:29.195708 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:53:29.253484 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1369/2757] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:53:29 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:53:29.266939 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13aa046-b10c-44ad-bdff-6964439366d7 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:29.268456 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-c13aa046-b10c-44ad-bdff-6964439366d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:29.271445 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bd1012-e93c-4cb2-8e43-060a277df001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:29.272039 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bd1012-e93c-4cb2-8e43-060a277df001 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:29.272327 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bd1012-e93c-4cb2-8e43-060a277df001 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:29.272633 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bd1012-e93c-4cb2-8e43-060a277df001 admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:29.275337 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bd1012-e93c-4cb2-8e43-060a277df001 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:29.284115 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bd1012-e93c-4cb2-8e43-060a277df001 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:29.294779 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1389/2758] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:53:29 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:53:29.299067 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bd1012-e93c-4cb2-8e43-060a277df001 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:29.317272 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bd1012-e93c-4cb2-8e43-060a277df001 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:29.317547 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:53:29.338598 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85037869-0a72-4be2-ba52-77c470497ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:29.339290 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85037869-0a72-4be2-ba52-77c470497ef4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:29.339816 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85037869-0a72-4be2-ba52-77c470497ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:29.340291 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85037869-0a72-4be2-ba52-77c470497ef4 admin admin] PATH_INFO: `/v3/projects/9c12be764c1d4f3b969f2ac13700b25e/users/557bc82adea845c0a33294cedb48e68a/roles/486144ad1bcd417e86acdbc1d14327bf` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:29.366597 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13aa046-b10c-44ad-bdff-6964439366d7 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:29.373429 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd28a1f-6170-431c-972b-19ba399f3f5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36639dce4f97412f89d71367f650b901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:29.375042 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd28a1f-6170-431c-972b-19ba399f3f5f service cinder] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:29.375042 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd28a1f-6170-431c-972b-19ba399f3f5f service cinder] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:29.375344 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd28a1f-6170-431c-972b-19ba399f3f5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:29.376241 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85037869-0a72-4be2-ba52-77c470497ef4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c12be764c1d4f3b969f2ac13700b25e, user_id=557bc82adea845c0a33294cedb48e68a, role_id=486144ad1bcd417e86acdbc1d14327bf)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:29.379058 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85037869-0a72-4be2-ba52-77c470497ef4 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:29.390230 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd28a1f-6170-431c-972b-19ba399f3f5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:29.391691 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd28a1f-6170-431c-972b-19ba399f3f5f service cinder] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:29.402633 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1370/2759] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:53:29 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:53:29.462383 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1390/2760] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:53:29 2022] PUT /identity/v3/projects/9c12be764c1d4f3b969f2ac13700b25e/users/557bc82adea845c0a33294cedb48e68a/roles/486144ad1bcd417e86acdbc1d14327bf => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:53:29.466802 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd28a1f-6170-431c-972b-19ba399f3f5f service cinder] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:29.477225 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd28a1f-6170-431c-972b-19ba399f3f5f service cinder] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:29.543508 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8201d161-43b1-4906-8feb-ca747d00cde0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:29.544139 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8201d161-43b1-4906-8feb-ca747d00cde0 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:29.544430 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8201d161-43b1-4906-8feb-ca747d00cde0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:29.544705 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8201d161-43b1-4906-8feb-ca747d00cde0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:29.545389 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8201d161-43b1-4906-8feb-ca747d00cde0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:29.548100 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8201d161-43b1-4906-8feb-ca747d00cde0 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:29.565499 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85037869-0a72-4be2-ba52-77c470497ef4 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:29.573429 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1371/2761] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:53:29 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:53:29.580622 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85037869-0a72-4be2-ba52-77c470497ef4 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:29.587051 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5d8159-4d96-4b08-b6ea-5da9477da204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:29.588288 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5d8159-4d96-4b08-b6ea-5da9477da204 admin admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:29.588827 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5d8159-4d96-4b08-b6ea-5da9477da204 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:29.589142 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5d8159-4d96-4b08-b6ea-5da9477da204 admin admin] PATH_INFO: `/v3/projects/9c12be764c1d4f3b969f2ac13700b25e/users/557bc82adea845c0a33294cedb48e68a/roles/486144ad1bcd417e86acdbc1d14327bf` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:29.591757 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5d8159-4d96-4b08-b6ea-5da9477da204 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c12be764c1d4f3b969f2ac13700b25e, user_id=557bc82adea845c0a33294cedb48e68a, role_id=486144ad1bcd417e86acdbc1d14327bf)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:29.594713 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5d8159-4d96-4b08-b6ea-5da9477da204 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:29.606355 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1391/2762] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:53:29 2022] PUT /identity/v3/projects/9c12be764c1d4f3b969f2ac13700b25e/users/557bc82adea845c0a33294cedb48e68a/roles/486144ad1bcd417e86acdbc1d14327bf => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:53:29.617877 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc58b91-14d0-4a51-a74a-00d25f392f55 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:29.618822 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc58b91-14d0-4a51-a74a-00d25f392f55 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:29.620646 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc58b91-14d0-4a51-a74a-00d25f392f55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:29.658407 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-7dc58b91-14d0-4a51-a74a-00d25f392f55 None None] MFA Rules not processed for user `557bc82adea845c0a33294cedb48e68a`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:53:29.659653 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-7dc58b91-14d0-4a51-a74a-00d25f392f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:29.769866 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1372/2763] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:53:29 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:53:29.832215 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5d8159-4d96-4b08-b6ea-5da9477da204 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:29.834138 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-cf5d8159-4d96-4b08-b6ea-5da9477da204 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:29.919473 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5d8159-4d96-4b08-b6ea-5da9477da204 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:29.924004 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7af6c76-29c5-41ae-82ed-4c73008b4427 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36639dce4f97412f89d71367f650b901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:29.925071 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7af6c76-29c5-41ae-82ed-4c73008b4427 service cinder] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:29.925591 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7af6c76-29c5-41ae-82ed-4c73008b4427 service cinder] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:29.925959 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7af6c76-29c5-41ae-82ed-4c73008b4427 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:29.992973 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7af6c76-29c5-41ae-82ed-4c73008b4427 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:29.995205 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7af6c76-29c5-41ae-82ed-4c73008b4427 service cinder] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:30.005397 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1392/2764] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:53:29 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:53:31.570327 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc58b91-14d0-4a51-a74a-00d25f392f55 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:31.625043 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=100 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:53:31.646626 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc58b91-14d0-4a51-a74a-00d25f392f55 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:31.652448 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3b6950-4fcd-4bb5-9465-0d15aded54f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:31.653244 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b6950-4fcd-4bb5-9465-0d15aded54f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:31.653694 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b6950-4fcd-4bb5-9465-0d15aded54f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:31.654060 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b6950-4fcd-4bb5-9465-0d15aded54f6 admin admin] PATH_INFO: `/v3/users/d1996f7e94a749cdbd9bcfdfb6a61f04` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:31.655285 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3b6950-4fcd-4bb5-9465-0d15aded54f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1996f7e94a749cdbd9bcfdfb6a61f04)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:31.657341 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3b6950-4fcd-4bb5-9465-0d15aded54f6 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:31.890952 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-cb3b6950-4fcd-4bb5-9465-0d15aded54f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1996f7e94a749cdbd9bcfdfb6a61f04'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:31.891723 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-cb3b6950-4fcd-4bb5-9465-0d15aded54f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1996f7e94a749cdbd9bcfdfb6a61f04'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:31.892415 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-cb3b6950-4fcd-4bb5-9465-0d15aded54f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1996f7e94a749cdbd9bcfdfb6a61f04'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:31.901211 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-cb3b6950-4fcd-4bb5-9465-0d15aded54f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1996f7e94a749cdbd9bcfdfb6a61f04'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:31.976177 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-cb3b6950-4fcd-4bb5-9465-0d15aded54f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1996f7e94a749cdbd9bcfdfb6a61f04'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:32.011386 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1373/2765] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:53:31 2022] DELETE /identity/v3/users/d1996f7e94a749cdbd9bcfdfb6a61f04 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:53:32.017774 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95608f0-8f46-46f4-9304-550d50c88e3e None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:32.018366 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95608f0-8f46-46f4-9304-550d50c88e3e None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:32.018468 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95608f0-8f46-46f4-9304-550d50c88e3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:32.040910 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-d95608f0-8f46-46f4-9304-550d50c88e3e None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:53:32.042328 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-d95608f0-8f46-46f4-9304-550d50c88e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:32.063673 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:53:32.115729 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1393/2766] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:53:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:53:32.128980 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3b6950-4fcd-4bb5-9465-0d15aded54f6 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:32.132177 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-cb3b6950-4fcd-4bb5-9465-0d15aded54f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:32.195391 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3b6950-4fcd-4bb5-9465-0d15aded54f6 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:32.199817 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59abb499-f779-400a-a0fc-16704c8e922b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:32.200437 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59abb499-f779-400a-a0fc-16704c8e922b service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:32.200799 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59abb499-f779-400a-a0fc-16704c8e922b service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:32.201070 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59abb499-f779-400a-a0fc-16704c8e922b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:32.211625 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59abb499-f779-400a-a0fc-16704c8e922b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:32.216783 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59abb499-f779-400a-a0fc-16704c8e922b service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:32.224919 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1374/2767] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:53:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:53:32.948842 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95608f0-8f46-46f4-9304-550d50c88e3e None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:32.950407 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-d95608f0-8f46-46f4-9304-550d50c88e3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:32.963388 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95608f0-8f46-46f4-9304-550d50c88e3e None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:32.968324 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f809c7fc-b240-4270-8be3-b5fd37ecdb3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:32.969761 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f809c7fc-b240-4270-8be3-b5fd37ecdb3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:32.969761 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f809c7fc-b240-4270-8be3-b5fd37ecdb3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:32.969761 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f809c7fc-b240-4270-8be3-b5fd37ecdb3a admin admin] PATH_INFO: `/v3/projects/0f8d0d85723a4832897bebcdb48327bd` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:32.970808 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f809c7fc-b240-4270-8be3-b5fd37ecdb3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f8d0d85723a4832897bebcdb48327bd)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:32.972748 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f809c7fc-b240-4270-8be3-b5fd37ecdb3a admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:33.092794 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-f809c7fc-b240-4270-8be3-b5fd37ecdb3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f8d0d85723a4832897bebcdb48327bd', 'request_id': 'req-f809c7fc-b240-4270-8be3-b5fd37ecdb3a'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:33.192702 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-f809c7fc-b240-4270-8be3-b5fd37ecdb3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f8d0d85723a4832897bebcdb48327bd', 'request_id': 'req-f809c7fc-b240-4270-8be3-b5fd37ecdb3a'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:33.199911 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-f809c7fc-b240-4270-8be3-b5fd37ecdb3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f8d0d85723a4832897bebcdb48327bd', 'request_id': 'req-f809c7fc-b240-4270-8be3-b5fd37ecdb3a'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:33.211183 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-f809c7fc-b240-4270-8be3-b5fd37ecdb3a admin admin] The token cache is being invalidate because project 0f8d0d85723a4832897bebcdb48327bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81723) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:53:33.211473 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-f809c7fc-b240-4270-8be3-b5fd37ecdb3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:33.212124 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-f809c7fc-b240-4270-8be3-b5fd37ecdb3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:33.214841 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1394/2768] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:53:32 2022] DELETE /identity/v3/projects/0f8d0d85723a4832897bebcdb48327bd => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:53:33.218955 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59abb499-f779-400a-a0fc-16704c8e922b service neutron] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:33.230028 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59abb499-f779-400a-a0fc-16704c8e922b service neutron] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:33.291679 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09275514-67b1-4562-a714-430bf4c33d21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:33.292744 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09275514-67b1-4562-a714-430bf4c33d21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:33.293030 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09275514-67b1-4562-a714-430bf4c33d21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:33.293375 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09275514-67b1-4562-a714-430bf4c33d21 admin admin] PATH_INFO: `/v3/users/8378987fa33b45ccaf9350acb1516857` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:33.294531 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09275514-67b1-4562-a714-430bf4c33d21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8378987fa33b45ccaf9350acb1516857)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:33.296521 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09275514-67b1-4562-a714-430bf4c33d21 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:33.306982 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:53:33.487747 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-09275514-67b1-4562-a714-430bf4c33d21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8378987fa33b45ccaf9350acb1516857'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:33.488491 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-09275514-67b1-4562-a714-430bf4c33d21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8378987fa33b45ccaf9350acb1516857'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:33.489138 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-09275514-67b1-4562-a714-430bf4c33d21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8378987fa33b45ccaf9350acb1516857'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:33.496755 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-09275514-67b1-4562-a714-430bf4c33d21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8378987fa33b45ccaf9350acb1516857'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:33.586955 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-09275514-67b1-4562-a714-430bf4c33d21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8378987fa33b45ccaf9350acb1516857'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:33.609146 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1375/2769] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:53:33 2022] DELETE /identity/v3/users/8378987fa33b45ccaf9350acb1516857 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:53:34.098962 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f809c7fc-b240-4270-8be3-b5fd37ecdb3a admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:34.100711 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-f809c7fc-b240-4270-8be3-b5fd37ecdb3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:34.117653 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f809c7fc-b240-4270-8be3-b5fd37ecdb3a admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:34.187935 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e813a6d6-f506-4339-8873-ec3a19401379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:34.188690 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e813a6d6-f506-4339-8873-ec3a19401379 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:34.188971 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e813a6d6-f506-4339-8873-ec3a19401379 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:34.189349 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e813a6d6-f506-4339-8873-ec3a19401379 admin admin] PATH_INFO: `/v3/projects/0ba95bda3c6f46a5b49d5c367b21606b` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:34.190451 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e813a6d6-f506-4339-8873-ec3a19401379 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ba95bda3c6f46a5b49d5c367b21606b)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:34.192462 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e813a6d6-f506-4339-8873-ec3a19401379 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:34.357057 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-e813a6d6-f506-4339-8873-ec3a19401379 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ba95bda3c6f46a5b49d5c367b21606b', 'request_id': 'req-e813a6d6-f506-4339-8873-ec3a19401379'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:34.360090 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:53:34.455919 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-e813a6d6-f506-4339-8873-ec3a19401379 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ba95bda3c6f46a5b49d5c367b21606b', 'request_id': 'req-e813a6d6-f506-4339-8873-ec3a19401379'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:34.463644 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-e813a6d6-f506-4339-8873-ec3a19401379 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ba95bda3c6f46a5b49d5c367b21606b', 'request_id': 'req-e813a6d6-f506-4339-8873-ec3a19401379'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:34.474880 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-e813a6d6-f506-4339-8873-ec3a19401379 admin admin] The token cache is being invalidate because project 0ba95bda3c6f46a5b49d5c367b21606b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81723) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:53:34.475083 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-e813a6d6-f506-4339-8873-ec3a19401379 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:34.475794 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-e813a6d6-f506-4339-8873-ec3a19401379 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:34.477792 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1395/2770] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:53:34 2022] DELETE /identity/v3/projects/0ba95bda3c6f46a5b49d5c367b21606b => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:53:34.488009 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff93bc11-82cf-4b56-a3ef-101ea0ffe02e None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:34.490483 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff93bc11-82cf-4b56-a3ef-101ea0ffe02e None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:34.490771 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff93bc11-82cf-4b56-a3ef-101ea0ffe02e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:34.510816 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-ff93bc11-82cf-4b56-a3ef-101ea0ffe02e None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:53:34.512130 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-ff93bc11-82cf-4b56-a3ef-101ea0ffe02e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:34.573649 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1376/2771] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:53:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:53:34.582098 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61d368e-4e95-45a7-b553-02918ad7d03e None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:34.582378 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61d368e-4e95-45a7-b553-02918ad7d03e None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:34.582689 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61d368e-4e95-45a7-b553-02918ad7d03e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:34.617487 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-d61d368e-4e95-45a7-b553-02918ad7d03e None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:53:34.618988 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-d61d368e-4e95-45a7-b553-02918ad7d03e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:34.628403 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1396/2772] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:53:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:53:34.632802 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff93bc11-82cf-4b56-a3ef-101ea0ffe02e None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:34.655517 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff93bc11-82cf-4b56-a3ef-101ea0ffe02e None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:34.660450 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb181753-de4f-4316-89f8-549f7fda0f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:34.661070 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb181753-de4f-4316-89f8-549f7fda0f23 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:34.661391 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb181753-de4f-4316-89f8-549f7fda0f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:34.661664 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb181753-de4f-4316-89f8-549f7fda0f23 admin admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:34.662505 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb181753-de4f-4316-89f8-549f7fda0f23 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:53:34.662809 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb181753-de4f-4316-89f8-549f7fda0f23 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:34.664466 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb181753-de4f-4316-89f8-549f7fda0f23 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:34.678006 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1377/2773] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 24 06:53:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:53:34.687094 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61d368e-4e95-45a7-b553-02918ad7d03e None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:34.697069 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61d368e-4e95-45a7-b553-02918ad7d03e None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:34.702121 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b722e49-a0bd-436a-882c-513cc548b983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:34.702727 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b722e49-a0bd-436a-882c-513cc548b983 admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:34.703004 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b722e49-a0bd-436a-882c-513cc548b983 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:34.703280 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b722e49-a0bd-436a-882c-513cc548b983 admin admin] PATH_INFO: `/v3/projects` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:34.703903 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b722e49-a0bd-436a-882c-513cc548b983 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:34.705945 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b722e49-a0bd-436a-882c-513cc548b983 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:34.871007 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1397/2774] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:53:34 2022] POST /identity/v3/projects => generated 367 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:53:34.875363 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb181753-de4f-4316-89f8-549f7fda0f23 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:34.891126 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb181753-de4f-4316-89f8-549f7fda0f23 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:34.977794 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f27130-a431-473f-bf46-4d089a9db47e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:34.978513 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f27130-a431-473f-bf46-4d089a9db47e admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:34.978894 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f27130-a431-473f-bf46-4d089a9db47e admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:34.979356 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f27130-a431-473f-bf46-4d089a9db47e admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:34.980152 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f27130-a431-473f-bf46-4d089a9db47e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:34.982200 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f27130-a431-473f-bf46-4d089a9db47e admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:35.046806 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1378/2775] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:53:34 2022] POST /identity/v3/users => generated 284 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:53:35.050910 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b722e49-a0bd-436a-882c-513cc548b983 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:35.066036 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b722e49-a0bd-436a-882c-513cc548b983 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:35.076616 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dba432-c06b-4709-b045-07d1fff8ecc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:35.077769 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dba432-c06b-4709-b045-07d1fff8ecc6 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:35.078314 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dba432-c06b-4709-b045-07d1fff8ecc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:35.078868 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dba432-c06b-4709-b045-07d1fff8ecc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:35.080061 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dba432-c06b-4709-b045-07d1fff8ecc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:35.082953 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dba432-c06b-4709-b045-07d1fff8ecc6 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:35.102312 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1398/2776] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:53:35 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:53:35.111864 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f27130-a431-473f-bf46-4d089a9db47e admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:35.121104 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:53:35.128902 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f27130-a431-473f-bf46-4d089a9db47e admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:35.136363 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eff51d-7aa8-4fc3-81ac-04024b6722f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:35.137765 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eff51d-7aa8-4fc3-81ac-04024b6722f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:35.138049 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eff51d-7aa8-4fc3-81ac-04024b6722f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:35.138325 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eff51d-7aa8-4fc3-81ac-04024b6722f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:35.139967 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eff51d-7aa8-4fc3-81ac-04024b6722f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:35.141218 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eff51d-7aa8-4fc3-81ac-04024b6722f5 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:35.156895 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1379/2777] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:53:35 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:53:35.161417 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dba432-c06b-4709-b045-07d1fff8ecc6 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:35.188345 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dba432-c06b-4709-b045-07d1fff8ecc6 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:35.200731 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5965d308-f87b-4579-9d37-213dc4f7208e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:35.201745 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5965d308-f87b-4579-9d37-213dc4f7208e admin admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:35.202031 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5965d308-f87b-4579-9d37-213dc4f7208e admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:35.202312 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5965d308-f87b-4579-9d37-213dc4f7208e admin admin] PATH_INFO: `/v3/projects/393e217bb3e94cceafe51f296bab1b0e/users/c0731d2d802a4149a3f46c1ef15f809b/roles/e1f9b167c51c4a388cbc9594db8a465d` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:35.231100 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5965d308-f87b-4579-9d37-213dc4f7208e admin admin] RBAC: Authorizing `identity:create_grant(project_id=393e217bb3e94cceafe51f296bab1b0e, user_id=c0731d2d802a4149a3f46c1ef15f809b, role_id=e1f9b167c51c4a388cbc9594db8a465d)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:35.234163 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5965d308-f87b-4579-9d37-213dc4f7208e admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:35.308997 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1399/2778] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:53:35 2022] PUT /identity/v3/projects/393e217bb3e94cceafe51f296bab1b0e/users/c0731d2d802a4149a3f46c1ef15f809b/roles/e1f9b167c51c4a388cbc9594db8a465d => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:53:35.311033 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8516e99c-0e78-4721-8dda-684fe5d7f765 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:35.311560 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8516e99c-0e78-4721-8dda-684fe5d7f765 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:35.312481 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8516e99c-0e78-4721-8dda-684fe5d7f765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:35.344543 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-8516e99c-0e78-4721-8dda-684fe5d7f765 None None] MFA Rules not processed for user `c0731d2d802a4149a3f46c1ef15f809b`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:53:35.346224 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-8516e99c-0e78-4721-8dda-684fe5d7f765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:35.430228 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1380/2779] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:53:35 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:53:35.435468 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5965d308-f87b-4579-9d37-213dc4f7208e admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:35.513925 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5965d308-f87b-4579-9d37-213dc4f7208e admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:35.519017 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67706883-099d-4288-a39a-a0dfe1306e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:35.519722 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67706883-099d-4288-a39a-a0dfe1306e42 admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:35.520223 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67706883-099d-4288-a39a-a0dfe1306e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:35.520503 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67706883-099d-4288-a39a-a0dfe1306e42 admin admin] PATH_INFO: `/v3/projects` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:35.521155 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67706883-099d-4288-a39a-a0dfe1306e42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:35.531282 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67706883-099d-4288-a39a-a0dfe1306e42 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:35.685288 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1400/2780] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:53:35 2022] POST /identity/v3/projects => generated 367 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:53:35.689340 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8516e99c-0e78-4721-8dda-684fe5d7f765 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:35.714630 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8516e99c-0e78-4721-8dda-684fe5d7f765 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:35.835440 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e569a4-8106-41a0-9f49-e51610bbffd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:35.836846 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e569a4-8106-41a0-9f49-e51610bbffd5 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:35.839134 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e569a4-8106-41a0-9f49-e51610bbffd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:35.839497 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e569a4-8106-41a0-9f49-e51610bbffd5 admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:35.846336 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e569a4-8106-41a0-9f49-e51610bbffd5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:35.846336 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e569a4-8106-41a0-9f49-e51610bbffd5 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:35.972009 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1381/2781] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:53:35 2022] POST /identity/v3/users => generated 290 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:53:35.975855 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67706883-099d-4288-a39a-a0dfe1306e42 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:35.988189 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67706883-099d-4288-a39a-a0dfe1306e42 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:35.993415 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b9b787-ec57-4565-a545-afb216677940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:35.994145 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b9b787-ec57-4565-a545-afb216677940 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:35.994554 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b9b787-ec57-4565-a545-afb216677940 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:35.995036 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b9b787-ec57-4565-a545-afb216677940 admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:35.995987 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b9b787-ec57-4565-a545-afb216677940 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:36.001228 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b9b787-ec57-4565-a545-afb216677940 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:36.012470 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1401/2782] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:53:35 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:53:36.016825 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e569a4-8106-41a0-9f49-e51610bbffd5 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:36.028107 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e569a4-8106-41a0-9f49-e51610bbffd5 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:36.033697 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5532efe4-60ae-492c-9204-2ef82b996f3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:36.034354 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5532efe4-60ae-492c-9204-2ef82b996f3f admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:36.034640 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5532efe4-60ae-492c-9204-2ef82b996f3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:36.034914 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5532efe4-60ae-492c-9204-2ef82b996f3f admin admin] PATH_INFO: `/v3/projects/ded0738b65ba4b7fba95a93b0a7a7f54/users/07a55664fd4640c5bdb47ba307a57dcf/roles/486144ad1bcd417e86acdbc1d14327bf` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:36.051292 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5532efe4-60ae-492c-9204-2ef82b996f3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ded0738b65ba4b7fba95a93b0a7a7f54, user_id=07a55664fd4640c5bdb47ba307a57dcf, role_id=486144ad1bcd417e86acdbc1d14327bf)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:36.054165 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5532efe4-60ae-492c-9204-2ef82b996f3f admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:36.153835 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1382/2783] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:53:36 2022] PUT /identity/v3/projects/ded0738b65ba4b7fba95a93b0a7a7f54/users/07a55664fd4640c5bdb47ba307a57dcf/roles/486144ad1bcd417e86acdbc1d14327bf => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:53:36.157929 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b9b787-ec57-4565-a545-afb216677940 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:36.171002 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b9b787-ec57-4565-a545-afb216677940 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:36.224012 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:53:36.246370 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3c229b-be67-4358-b64c-3676a3fe8da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:36.246370 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3c229b-be67-4358-b64c-3676a3fe8da7 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:36.246370 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3c229b-be67-4358-b64c-3676a3fe8da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:36.246370 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3c229b-be67-4358-b64c-3676a3fe8da7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:36.246915 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3c229b-be67-4358-b64c-3676a3fe8da7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:36.246915 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3c229b-be67-4358-b64c-3676a3fe8da7 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:36.253668 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1402/2784] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:53:36 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:53:36.257970 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5532efe4-60ae-492c-9204-2ef82b996f3f admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:36.270333 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5532efe4-60ae-492c-9204-2ef82b996f3f admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:36.276497 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6258e9-cdee-4b92-9234-00140db546b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:36.277614 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6258e9-cdee-4b92-9234-00140db546b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:36.277753 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6258e9-cdee-4b92-9234-00140db546b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:36.277939 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6258e9-cdee-4b92-9234-00140db546b5 admin admin] PATH_INFO: `/v3/projects/ded0738b65ba4b7fba95a93b0a7a7f54/users/07a55664fd4640c5bdb47ba307a57dcf/roles/486144ad1bcd417e86acdbc1d14327bf` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:36.280323 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6258e9-cdee-4b92-9234-00140db546b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ded0738b65ba4b7fba95a93b0a7a7f54, user_id=07a55664fd4640c5bdb47ba307a57dcf, role_id=486144ad1bcd417e86acdbc1d14327bf)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:36.283110 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6258e9-cdee-4b92-9234-00140db546b5 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:36.295025 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1383/2785] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:53:36 2022] PUT /identity/v3/projects/ded0738b65ba4b7fba95a93b0a7a7f54/users/07a55664fd4640c5bdb47ba307a57dcf/roles/486144ad1bcd417e86acdbc1d14327bf => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:53:36.305412 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d8d48-68a2-4a90-9d53-8a2a1e41b1d9 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:36.305858 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d8d48-68a2-4a90-9d53-8a2a1e41b1d9 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:36.306127 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d8d48-68a2-4a90-9d53-8a2a1e41b1d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:36.349166 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-4e9d8d48-68a2-4a90-9d53-8a2a1e41b1d9 None None] MFA Rules not processed for user `07a55664fd4640c5bdb47ba307a57dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:53:36.349166 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-4e9d8d48-68a2-4a90-9d53-8a2a1e41b1d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:36.436734 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1403/2786] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:53:36 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:53:36.451731 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6258e9-cdee-4b92-9234-00140db546b5 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:36.453495 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-8a6258e9-cdee-4b92-9234-00140db546b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:36.499256 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:53:36.516940 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6258e9-cdee-4b92-9234-00140db546b5 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:36.520893 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c7358e-3d58-4496-b80b-c32a5cf945c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36639dce4f97412f89d71367f650b901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:36.521646 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c7358e-3d58-4496-b80b-c32a5cf945c5 service cinder] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:36.521924 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c7358e-3d58-4496-b80b-c32a5cf945c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:36.522196 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c7358e-3d58-4496-b80b-c32a5cf945c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:36.533336 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c7358e-3d58-4496-b80b-c32a5cf945c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:36.534983 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c7358e-3d58-4496-b80b-c32a5cf945c5 service cinder] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:36.546708 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1384/2787] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:53:36 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:53:36.808925 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9d8d48-68a2-4a90-9d53-8a2a1e41b1d9 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:36.810403 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-4e9d8d48-68a2-4a90-9d53-8a2a1e41b1d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:36.828194 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9d8d48-68a2-4a90-9d53-8a2a1e41b1d9 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:36.828194 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acbb8f5-f4b2-4dfa-8cce-37828f5a3730 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36639dce4f97412f89d71367f650b901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:36.828194 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acbb8f5-f4b2-4dfa-8cce-37828f5a3730 service cinder] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:36.828194 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acbb8f5-f4b2-4dfa-8cce-37828f5a3730 service cinder] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:36.828606 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acbb8f5-f4b2-4dfa-8cce-37828f5a3730 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:36.896236 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c7358e-3d58-4496-b80b-c32a5cf945c5 service cinder] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:36.906470 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c7358e-3d58-4496-b80b-c32a5cf945c5 service cinder] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:36.910620 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3dc8f3-d8b7-44de-bf9a-ca103cd44ef5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36639dce4f97412f89d71367f650b901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:36.911023 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3dc8f3-d8b7-44de-bf9a-ca103cd44ef5 service cinder] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:36.911395 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3dc8f3-d8b7-44de-bf9a-ca103cd44ef5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:36.911670 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3dc8f3-d8b7-44de-bf9a-ca103cd44ef5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:36.913482 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-dd3dc8f3-d8b7-44de-bf9a-ca103cd44ef5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:36.927423 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acbb8f5-f4b2-4dfa-8cce-37828f5a3730 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:36.929260 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acbb8f5-f4b2-4dfa-8cce-37828f5a3730 service cinder] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:36.941099 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1404/2788] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:53:36 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:53:36.981977 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3dc8f3-d8b7-44de-bf9a-ca103cd44ef5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:36.983404 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3dc8f3-d8b7-44de-bf9a-ca103cd44ef5 service cinder] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:36.994319 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1385/2789] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:53:36 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:53:42.827836 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acbb8f5-f4b2-4dfa-8cce-37828f5a3730 service cinder] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:42.829039 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-1acbb8f5-f4b2-4dfa-8cce-37828f5a3730 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:42.889543 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acbb8f5-f4b2-4dfa-8cce-37828f5a3730 service cinder] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:42.893271 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1471d03d-8b6b-4344-9b13-698e5fb060d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:42.893921 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1471d03d-8b6b-4344-9b13-698e5fb060d8 service neutron] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:42.894199 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1471d03d-8b6b-4344-9b13-698e5fb060d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:42.894522 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1471d03d-8b6b-4344-9b13-698e5fb060d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:42.896366 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-1471d03d-8b6b-4344-9b13-698e5fb060d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:42.928454 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:53:42.969411 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1471d03d-8b6b-4344-9b13-698e5fb060d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:42.970981 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1471d03d-8b6b-4344-9b13-698e5fb060d8 service neutron] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:42.980815 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1405/2790] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:53:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:53:43.527671 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3dc8f3-d8b7-44de-bf9a-ca103cd44ef5 service cinder] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:43.531314 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-dd3dc8f3-d8b7-44de-bf9a-ca103cd44ef5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:43.630850 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3dc8f3-d8b7-44de-bf9a-ca103cd44ef5 service cinder] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:43.638342 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ba608d-1376-41ab-bab0-9e9195b1d5bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4f49c1b08c460f8ba6c451c9c92c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:43.638342 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ba608d-1376-41ab-bab0-9e9195b1d5bc service nova] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:43.638680 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ba608d-1376-41ab-bab0-9e9195b1d5bc service nova] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:43.638680 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ba608d-1376-41ab-bab0-9e9195b1d5bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:43.641169 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-a2ba608d-1376-41ab-bab0-9e9195b1d5bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:43.655703 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ba608d-1376-41ab-bab0-9e9195b1d5bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:43.657700 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ba608d-1376-41ab-bab0-9e9195b1d5bc service nova] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:43.668977 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1386/2791] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:53:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:53:44.007408 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1471d03d-8b6b-4344-9b13-698e5fb060d8 service neutron] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:44.020486 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1471d03d-8b6b-4344-9b13-698e5fb060d8 service neutron] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:44.022911 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c0a256-a2b7-4230-a2b8-5d393b257491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:44.023556 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c0a256-a2b7-4230-a2b8-5d393b257491 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:44.023828 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c0a256-a2b7-4230-a2b8-5d393b257491 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:44.024099 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c0a256-a2b7-4230-a2b8-5d393b257491 admin admin] PATH_INFO: `/v3/users/c0731d2d802a4149a3f46c1ef15f809b` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:44.025402 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c0a256-a2b7-4230-a2b8-5d393b257491 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0731d2d802a4149a3f46c1ef15f809b)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:44.030237 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c0a256-a2b7-4230-a2b8-5d393b257491 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:44.215416 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ba608d-1376-41ab-bab0-9e9195b1d5bc service nova] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:44.216828 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-a2ba608d-1376-41ab-bab0-9e9195b1d5bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:44.241196 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ba608d-1376-41ab-bab0-9e9195b1d5bc service nova] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:44.242634 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-86c0a256-a2b7-4230-a2b8-5d393b257491 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0731d2d802a4149a3f46c1ef15f809b'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:44.243362 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-86c0a256-a2b7-4230-a2b8-5d393b257491 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0731d2d802a4149a3f46c1ef15f809b'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:44.244061 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-86c0a256-a2b7-4230-a2b8-5d393b257491 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0731d2d802a4149a3f46c1ef15f809b'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:44.246189 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e3a72c-c5d5-4100-9218-fff236c3469a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:44.246830 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e3a72c-c5d5-4100-9218-fff236c3469a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:44.247103 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e3a72c-c5d5-4100-9218-fff236c3469a admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:44.247378 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e3a72c-c5d5-4100-9218-fff236c3469a admin admin] PATH_INFO: `/v3/users/d08b7dc4929045a0ac649bbbc9f46e3d` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:44.248716 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e3a72c-c5d5-4100-9218-fff236c3469a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d08b7dc4929045a0ac649bbbc9f46e3d)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:44.251205 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e3a72c-c5d5-4100-9218-fff236c3469a admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:44.306582 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-86c0a256-a2b7-4230-a2b8-5d393b257491 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0731d2d802a4149a3f46c1ef15f809b'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:44.320028 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-86c0a256-a2b7-4230-a2b8-5d393b257491 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0731d2d802a4149a3f46c1ef15f809b'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:44.338257 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1406/2792] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:53:44 2022] DELETE /identity/v3/users/c0731d2d802a4149a3f46c1ef15f809b => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:53:44.342751 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf112ac-ec12-4917-9e77-a0d885f321e2 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:44.343054 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf112ac-ec12-4917-9e77-a0d885f321e2 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:44.343376 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf112ac-ec12-4917-9e77-a0d885f321e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:44.363121 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-8cf112ac-ec12-4917-9e77-a0d885f321e2 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:53:44.364330 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-8cf112ac-ec12-4917-9e77-a0d885f321e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:44.429139 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1407/2793] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:53:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:53:44.441049 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf112ac-ec12-4917-9e77-a0d885f321e2 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:44.443131 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-8cf112ac-ec12-4917-9e77-a0d885f321e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:44.448591 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-78e3a72c-c5d5-4100-9218-fff236c3469a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd08b7dc4929045a0ac649bbbc9f46e3d'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:44.449294 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-78e3a72c-c5d5-4100-9218-fff236c3469a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd08b7dc4929045a0ac649bbbc9f46e3d'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:44.449992 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-78e3a72c-c5d5-4100-9218-fff236c3469a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd08b7dc4929045a0ac649bbbc9f46e3d'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:44.456548 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:53:44.463308 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-78e3a72c-c5d5-4100-9218-fff236c3469a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd08b7dc4929045a0ac649bbbc9f46e3d'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:44.484535 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:53:44.508110 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf112ac-ec12-4917-9e77-a0d885f321e2 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:44.511953 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30621117-f36b-4a2e-9149-345e81132420 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:44.512118 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-78e3a72c-c5d5-4100-9218-fff236c3469a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd08b7dc4929045a0ac649bbbc9f46e3d'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:44.512614 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30621117-f36b-4a2e-9149-345e81132420 service neutron] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:44.512859 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30621117-f36b-4a2e-9149-345e81132420 service neutron] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:44.513137 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30621117-f36b-4a2e-9149-345e81132420 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:44.515723 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-30621117-f36b-4a2e-9149-345e81132420 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:44.531406 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30621117-f36b-4a2e-9149-345e81132420 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:44.532858 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30621117-f36b-4a2e-9149-345e81132420 service neutron] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:44.535587 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1387/2794] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:53:44 2022] DELETE /identity/v3/users/d08b7dc4929045a0ac649bbbc9f46e3d => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:53:44.539920 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d063a0-a4bb-4148-8e7c-1f60a02e2182 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:44.540378 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d063a0-a4bb-4148-8e7c-1f60a02e2182 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:44.541139 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d063a0-a4bb-4148-8e7c-1f60a02e2182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:44.548057 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1408/2795] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:53:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:53:44.565112 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-95d063a0-a4bb-4148-8e7c-1f60a02e2182 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:53:44.565112 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-95d063a0-a4bb-4148-8e7c-1f60a02e2182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:44.669036 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1388/2796] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:53:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:53:44.698179 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30621117-f36b-4a2e-9149-345e81132420 service neutron] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:44.701516 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-30621117-f36b-4a2e-9149-345e81132420 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:44.726762 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30621117-f36b-4a2e-9149-345e81132420 service neutron] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:44.798384 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58a5230-0fb0-4892-8c3b-04a6279df6ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:44.799008 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58a5230-0fb0-4892-8c3b-04a6279df6ab service neutron] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:44.799285 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58a5230-0fb0-4892-8c3b-04a6279df6ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:44.799552 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58a5230-0fb0-4892-8c3b-04a6279df6ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:44.810696 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58a5230-0fb0-4892-8c3b-04a6279df6ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:44.812574 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58a5230-0fb0-4892-8c3b-04a6279df6ab service neutron] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:44.824440 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1409/2797] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:53:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:53:45.324988 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d063a0-a4bb-4148-8e7c-1f60a02e2182 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:45.326494 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-95d063a0-a4bb-4148-8e7c-1f60a02e2182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:45.340226 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d063a0-a4bb-4148-8e7c-1f60a02e2182 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:45.346066 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91fa738-6bb3-4fd9-ace5-71272cd8890e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:45.346835 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91fa738-6bb3-4fd9-ace5-71272cd8890e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:45.347121 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91fa738-6bb3-4fd9-ace5-71272cd8890e admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:45.347394 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91fa738-6bb3-4fd9-ace5-71272cd8890e admin admin] PATH_INFO: `/v3/projects/393e217bb3e94cceafe51f296bab1b0e` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:45.348702 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91fa738-6bb3-4fd9-ace5-71272cd8890e admin admin] RBAC: Authorizing `identity:delete_project(project_id=393e217bb3e94cceafe51f296bab1b0e)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:45.350688 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91fa738-6bb3-4fd9-ace5-71272cd8890e admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:45.382879 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58a5230-0fb0-4892-8c3b-04a6279df6ab service neutron] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:45.384368 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-e58a5230-0fb0-4892-8c3b-04a6279df6ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:45.398923 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58a5230-0fb0-4892-8c3b-04a6279df6ab service neutron] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:45.406330 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf0ada8-f98d-40e0-8924-264413567b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:45.406966 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf0ada8-f98d-40e0-8924-264413567b56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:45.407368 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf0ada8-f98d-40e0-8924-264413567b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:45.407659 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf0ada8-f98d-40e0-8924-264413567b56 admin admin] PATH_INFO: `/v3/projects/98be857d628b42e6b6fe8f2f45b7dd74` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:45.409150 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf0ada8-f98d-40e0-8924-264413567b56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98be857d628b42e6b6fe8f2f45b7dd74)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:45.411072 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf0ada8-f98d-40e0-8924-264413567b56 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:45.472210 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-a91fa738-6bb3-4fd9-ace5-71272cd8890e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '393e217bb3e94cceafe51f296bab1b0e', 'request_id': 'req-a91fa738-6bb3-4fd9-ace5-71272cd8890e'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:45.478435 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-a91fa738-6bb3-4fd9-ace5-71272cd8890e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '393e217bb3e94cceafe51f296bab1b0e', 'request_id': 'req-a91fa738-6bb3-4fd9-ace5-71272cd8890e'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:45.543133 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-ecf0ada8-f98d-40e0-8924-264413567b56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98be857d628b42e6b6fe8f2f45b7dd74', 'request_id': 'req-ecf0ada8-f98d-40e0-8924-264413567b56'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:45.555362 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-a91fa738-6bb3-4fd9-ace5-71272cd8890e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '393e217bb3e94cceafe51f296bab1b0e', 'request_id': 'req-a91fa738-6bb3-4fd9-ace5-71272cd8890e'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:45.579354 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-a91fa738-6bb3-4fd9-ace5-71272cd8890e admin admin] The token cache is being invalidate because project 393e217bb3e94cceafe51f296bab1b0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:53:45.580573 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-a91fa738-6bb3-4fd9-ace5-71272cd8890e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:45.581732 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-a91fa738-6bb3-4fd9-ace5-71272cd8890e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:45.584529 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1389/2798] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:53:45 2022] DELETE /identity/v3/projects/393e217bb3e94cceafe51f296bab1b0e => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:53:45.589618 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91fa738-6bb3-4fd9-ace5-71272cd8890e admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:45.592237 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-a91fa738-6bb3-4fd9-ace5-71272cd8890e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:45.683825 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-ecf0ada8-f98d-40e0-8924-264413567b56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98be857d628b42e6b6fe8f2f45b7dd74', 'request_id': 'req-ecf0ada8-f98d-40e0-8924-264413567b56'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:45.688142 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:53:45.695259 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-ecf0ada8-f98d-40e0-8924-264413567b56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98be857d628b42e6b6fe8f2f45b7dd74', 'request_id': 'req-ecf0ada8-f98d-40e0-8924-264413567b56'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:45.701908 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91fa738-6bb3-4fd9-ace5-71272cd8890e admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:45.709827 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-ecf0ada8-f98d-40e0-8924-264413567b56 admin admin] The token cache is being invalidate because project 98be857d628b42e6b6fe8f2f45b7dd74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81723) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:53:45.710416 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-ecf0ada8-f98d-40e0-8924-264413567b56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:45.711262 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-ecf0ada8-f98d-40e0-8924-264413567b56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:45.716722 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1410/2799] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:53:45 2022] DELETE /identity/v3/projects/98be857d628b42e6b6fe8f2f45b7dd74 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:53:45.717879 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf0ada8-f98d-40e0-8924-264413567b56 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:45.719884 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-ecf0ada8-f98d-40e0-8924-264413567b56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:45.769648 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb57f8f-8204-44f1-8fe7-dbfba400764a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:45.770917 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb57f8f-8204-44f1-8fe7-dbfba400764a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:45.771305 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb57f8f-8204-44f1-8fe7-dbfba400764a admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:45.771475 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb57f8f-8204-44f1-8fe7-dbfba400764a admin admin] PATH_INFO: `/v3/users/07a55664fd4640c5bdb47ba307a57dcf` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:45.772639 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb57f8f-8204-44f1-8fe7-dbfba400764a admin admin] RBAC: Authorizing `identity:delete_user(user_id=07a55664fd4640c5bdb47ba307a57dcf)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:45.776068 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb57f8f-8204-44f1-8fe7-dbfba400764a admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:45.793287 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf0ada8-f98d-40e0-8924-264413567b56 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:45.802326 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7d23bb-503e-47cd-b58c-dee1e5808640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:45.803731 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7d23bb-503e-47cd-b58c-dee1e5808640 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:45.804084 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7d23bb-503e-47cd-b58c-dee1e5808640 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:45.804434 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7d23bb-503e-47cd-b58c-dee1e5808640 admin admin] PATH_INFO: `/v3/users/557bc82adea845c0a33294cedb48e68a` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:45.805994 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7d23bb-503e-47cd-b58c-dee1e5808640 admin admin] RBAC: Authorizing `identity:delete_user(user_id=557bc82adea845c0a33294cedb48e68a)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:45.811035 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7d23bb-503e-47cd-b58c-dee1e5808640 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:45.812797 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:53:45.982870 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-9e7d23bb-503e-47cd-b58c-dee1e5808640 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '557bc82adea845c0a33294cedb48e68a'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:45.990234 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-9e7d23bb-503e-47cd-b58c-dee1e5808640 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '557bc82adea845c0a33294cedb48e68a'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:45.992030 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-9e7d23bb-503e-47cd-b58c-dee1e5808640 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '557bc82adea845c0a33294cedb48e68a'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:46.011500 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-8eb57f8f-8204-44f1-8fe7-dbfba400764a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07a55664fd4640c5bdb47ba307a57dcf'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:46.015323 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-8eb57f8f-8204-44f1-8fe7-dbfba400764a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07a55664fd4640c5bdb47ba307a57dcf'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:46.015323 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-8eb57f8f-8204-44f1-8fe7-dbfba400764a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07a55664fd4640c5bdb47ba307a57dcf'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:46.022245 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-8eb57f8f-8204-44f1-8fe7-dbfba400764a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07a55664fd4640c5bdb47ba307a57dcf'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:46.047681 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-9e7d23bb-503e-47cd-b58c-dee1e5808640 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '557bc82adea845c0a33294cedb48e68a'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:46.070611 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-9e7d23bb-503e-47cd-b58c-dee1e5808640 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '557bc82adea845c0a33294cedb48e68a'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:46.084790 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1411/2800] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:53:45 2022] DELETE /identity/v3/users/557bc82adea845c0a33294cedb48e68a => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:53:46.147156 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-8eb57f8f-8204-44f1-8fe7-dbfba400764a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07a55664fd4640c5bdb47ba307a57dcf'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:46.169277 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1390/2801] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:53:45 2022] DELETE /identity/v3/users/07a55664fd4640c5bdb47ba307a57dcf => generated 0 bytes in 581 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:53:46.402134 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7d23bb-503e-47cd-b58c-dee1e5808640 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:46.403607 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-9e7d23bb-503e-47cd-b58c-dee1e5808640 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:46.482739 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7d23bb-503e-47cd-b58c-dee1e5808640 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:46.560602 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da73f401-d2ee-43b7-820a-82904c1e8364 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:46.561994 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da73f401-d2ee-43b7-820a-82904c1e8364 service neutron] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:46.562846 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da73f401-d2ee-43b7-820a-82904c1e8364 service neutron] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:46.562846 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da73f401-d2ee-43b7-820a-82904c1e8364 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:46.565051 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-da73f401-d2ee-43b7-820a-82904c1e8364 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:46.653382 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:53:46.684947 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da73f401-d2ee-43b7-820a-82904c1e8364 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:46.687643 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da73f401-d2ee-43b7-820a-82904c1e8364 service neutron] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:46.701630 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1412/2802] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:53:46 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:53:46.963867 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb57f8f-8204-44f1-8fe7-dbfba400764a admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:46.965422 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-8eb57f8f-8204-44f1-8fe7-dbfba400764a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:47.077534 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb57f8f-8204-44f1-8fe7-dbfba400764a admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:47.085151 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30d11ed-aefa-414b-93c9-e2a2f196851e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:47.085937 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30d11ed-aefa-414b-93c9-e2a2f196851e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:47.086476 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30d11ed-aefa-414b-93c9-e2a2f196851e admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:47.086825 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30d11ed-aefa-414b-93c9-e2a2f196851e admin admin] PATH_INFO: `/v3/projects/9c12be764c1d4f3b969f2ac13700b25e` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:47.088585 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30d11ed-aefa-414b-93c9-e2a2f196851e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c12be764c1d4f3b969f2ac13700b25e)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:47.090845 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30d11ed-aefa-414b-93c9-e2a2f196851e admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:47.121262 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:53:47.223909 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-a30d11ed-aefa-414b-93c9-e2a2f196851e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c12be764c1d4f3b969f2ac13700b25e', 'request_id': 'req-a30d11ed-aefa-414b-93c9-e2a2f196851e'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:47.229852 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-a30d11ed-aefa-414b-93c9-e2a2f196851e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c12be764c1d4f3b969f2ac13700b25e', 'request_id': 'req-a30d11ed-aefa-414b-93c9-e2a2f196851e'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:47.253611 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da73f401-d2ee-43b7-820a-82904c1e8364 service neutron] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:47.253611 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-da73f401-d2ee-43b7-820a-82904c1e8364 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:47.264283 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da73f401-d2ee-43b7-820a-82904c1e8364 service neutron] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:47.303333 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-a30d11ed-aefa-414b-93c9-e2a2f196851e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c12be764c1d4f3b969f2ac13700b25e', 'request_id': 'req-a30d11ed-aefa-414b-93c9-e2a2f196851e'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:47.325089 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-a30d11ed-aefa-414b-93c9-e2a2f196851e admin admin] The token cache is being invalidate because project 9c12be764c1d4f3b969f2ac13700b25e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:53:47.325496 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-a30d11ed-aefa-414b-93c9-e2a2f196851e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:47.326270 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-a30d11ed-aefa-414b-93c9-e2a2f196851e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:47.328710 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1391/2803] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:53:46 2022] DELETE /identity/v3/projects/9c12be764c1d4f3b969f2ac13700b25e => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:53:47.334741 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b45215-aba2-46f9-8567-3227f63b8370 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:47.335224 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b45215-aba2-46f9-8567-3227f63b8370 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:47.335947 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b45215-aba2-46f9-8567-3227f63b8370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:47.363796 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-93b45215-aba2-46f9-8567-3227f63b8370 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:53:47.364141 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cd9630-562e-41f3-b52c-0415fef8abfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:47.366231 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-93b45215-aba2-46f9-8567-3227f63b8370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:47.367248 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cd9630-562e-41f3-b52c-0415fef8abfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:47.367703 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cd9630-562e-41f3-b52c-0415fef8abfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:47.368209 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cd9630-562e-41f3-b52c-0415fef8abfe admin admin] PATH_INFO: `/v3/projects/ded0738b65ba4b7fba95a93b0a7a7f54` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:47.369721 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cd9630-562e-41f3-b52c-0415fef8abfe admin admin] RBAC: Authorizing `identity:delete_project(project_id=ded0738b65ba4b7fba95a93b0a7a7f54)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:47.371920 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cd9630-562e-41f3-b52c-0415fef8abfe admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:47.376930 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1392/2804] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:53:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:53:47.384574 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ab73bd-edc8-4ef3-99ad-be08682f39cf None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:47.385073 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ab73bd-edc8-4ef3-99ad-be08682f39cf None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:47.385400 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ab73bd-edc8-4ef3-99ad-be08682f39cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:47.409921 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-44ab73bd-edc8-4ef3-99ad-be08682f39cf None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:53:47.411211 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-44ab73bd-edc8-4ef3-99ad-be08682f39cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:47.420693 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1393/2805] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:53:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:53:47.425019 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ab73bd-edc8-4ef3-99ad-be08682f39cf None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:47.435041 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ab73bd-edc8-4ef3-99ad-be08682f39cf None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:47.440759 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4999a10c-a7a0-4150-acd7-3639d087ef56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:47.441426 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4999a10c-a7a0-4150-acd7-3639d087ef56 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:47.441765 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4999a10c-a7a0-4150-acd7-3639d087ef56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:47.448950 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4999a10c-a7a0-4150-acd7-3639d087ef56 admin admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:47.448950 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4999a10c-a7a0-4150-acd7-3639d087ef56 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:53:47.448950 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4999a10c-a7a0-4150-acd7-3639d087ef56 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:47.448950 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4999a10c-a7a0-4150-acd7-3639d087ef56 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:47.462265 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1394/2806] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 24 06:53:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:53:47.467905 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4999a10c-a7a0-4150-acd7-3639d087ef56 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:47.482591 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4999a10c-a7a0-4150-acd7-3639d087ef56 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:47.490333 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-55cd9630-562e-41f3-b52c-0415fef8abfe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ded0738b65ba4b7fba95a93b0a7a7f54', 'request_id': 'req-55cd9630-562e-41f3-b52c-0415fef8abfe'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:47.552382 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fd16d5-2b54-4d86-87a1-d8140e96db63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:47.553079 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fd16d5-2b54-4d86-87a1-d8140e96db63 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:47.553397 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fd16d5-2b54-4d86-87a1-d8140e96db63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:47.554276 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fd16d5-2b54-4d86-87a1-d8140e96db63 admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:47.554412 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fd16d5-2b54-4d86-87a1-d8140e96db63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:47.556262 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fd16d5-2b54-4d86-87a1-d8140e96db63 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:47.569497 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-55cd9630-562e-41f3-b52c-0415fef8abfe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ded0738b65ba4b7fba95a93b0a7a7f54', 'request_id': 'req-55cd9630-562e-41f3-b52c-0415fef8abfe'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:47.577788 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-55cd9630-562e-41f3-b52c-0415fef8abfe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ded0738b65ba4b7fba95a93b0a7a7f54', 'request_id': 'req-55cd9630-562e-41f3-b52c-0415fef8abfe'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:47.591175 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-55cd9630-562e-41f3-b52c-0415fef8abfe admin admin] The token cache is being invalidate because project ded0738b65ba4b7fba95a93b0a7a7f54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81723) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:53:47.591473 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-55cd9630-562e-41f3-b52c-0415fef8abfe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:47.592158 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-55cd9630-562e-41f3-b52c-0415fef8abfe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:47.593847 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1413/2807] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:53:47 2022] DELETE /identity/v3/projects/ded0738b65ba4b7fba95a93b0a7a7f54 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:53:47.598842 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0048b4be-66ca-4a55-a2fe-f73bb6b255cb None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:47.599129 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0048b4be-66ca-4a55-a2fe-f73bb6b255cb None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:47.599406 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0048b4be-66ca-4a55-a2fe-f73bb6b255cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:47.630389 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-0048b4be-66ca-4a55-a2fe-f73bb6b255cb None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:53:47.630605 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-0048b4be-66ca-4a55-a2fe-f73bb6b255cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:47.645880 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1395/2808] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:53:47 2022] POST /identity/v3/projects => generated 373 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:53:47.649084 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fd16d5-2b54-4d86-87a1-d8140e96db63 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:47.649852 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1414/2809] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:53:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:53:47.651704 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-56fd16d5-2b54-4d86-87a1-d8140e96db63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:47.653432 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a66cdfb-698b-4a06-946d-06bd38efe3ca None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:47.653775 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a66cdfb-698b-4a06-946d-06bd38efe3ca None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:47.654091 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a66cdfb-698b-4a06-946d-06bd38efe3ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:47.691970 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-9a66cdfb-698b-4a06-946d-06bd38efe3ca None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:53:47.693265 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-9a66cdfb-698b-4a06-946d-06bd38efe3ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:47.786348 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fd16d5-2b54-4d86-87a1-d8140e96db63 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:47.794070 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd909e89-175e-4838-9868-65c5a3f6809b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:47.794795 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd909e89-175e-4838-9868-65c5a3f6809b admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:47.795133 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd909e89-175e-4838-9868-65c5a3f6809b admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:47.795468 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd909e89-175e-4838-9868-65c5a3f6809b admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:47.797804 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd909e89-175e-4838-9868-65c5a3f6809b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:47.799141 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd909e89-175e-4838-9868-65c5a3f6809b admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:47.801427 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1415/2810] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:53:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:53:47.805949 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a66cdfb-698b-4a06-946d-06bd38efe3ca None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:47.818118 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a66cdfb-698b-4a06-946d-06bd38efe3ca None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:47.824086 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c4bcce-d296-4356-a450-5b32ffacba04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:47.824931 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c4bcce-d296-4356-a450-5b32ffacba04 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:47.825379 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c4bcce-d296-4356-a450-5b32ffacba04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:47.825816 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c4bcce-d296-4356-a450-5b32ffacba04 admin admin] PATH_INFO: `/v3/domains` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:47.826622 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c4bcce-d296-4356-a450-5b32ffacba04 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81723) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:53:47.827091 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c4bcce-d296-4356-a450-5b32ffacba04 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:47.828911 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c4bcce-d296-4356-a450-5b32ffacba04 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:47.839375 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:53:47.851814 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1416/2811] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 24 06:53:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:53:47.855573 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c4bcce-d296-4356-a450-5b32ffacba04 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:47.861848 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1396/2812] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:53:47 2022] POST /identity/v3/users => generated 287 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:53:47.867562 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:DELETE=7,keystone:SELECT=88,keystone:INSERT=5 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:53:47.868076 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c4bcce-d296-4356-a450-5b32ffacba04 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:47.868458 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd909e89-175e-4838-9868-65c5a3f6809b admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:47.874527 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b2b71a-cbe9-49be-a189-2eb38d411ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:47.875304 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b2b71a-cbe9-49be-a189-2eb38d411ec9 admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:47.875596 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b2b71a-cbe9-49be-a189-2eb38d411ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:47.875968 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b2b71a-cbe9-49be-a189-2eb38d411ec9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:47.876726 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b2b71a-cbe9-49be-a189-2eb38d411ec9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:47.879398 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b2b71a-cbe9-49be-a189-2eb38d411ec9 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:47.882992 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd909e89-175e-4838-9868-65c5a3f6809b admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:47.892431 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937ce745-b93d-4b7a-ba5d-9a9c0eaaa1eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:47.893941 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937ce745-b93d-4b7a-ba5d-9a9c0eaaa1eb admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:47.894407 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937ce745-b93d-4b7a-ba5d-9a9c0eaaa1eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:47.894732 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937ce745-b93d-4b7a-ba5d-9a9c0eaaa1eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:47.899617 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937ce745-b93d-4b7a-ba5d-9a9c0eaaa1eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:47.901625 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937ce745-b93d-4b7a-ba5d-9a9c0eaaa1eb admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:47.922331 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1397/2813] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:53:47 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:53:47.923322 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937ce745-b93d-4b7a-ba5d-9a9c0eaaa1eb admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:47.934173 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937ce745-b93d-4b7a-ba5d-9a9c0eaaa1eb admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:47.939788 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d2f1be-b330-4c4c-ba87-2bd9ed0825c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:47.940425 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d2f1be-b330-4c4c-ba87-2bd9ed0825c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:47.940709 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d2f1be-b330-4c4c-ba87-2bd9ed0825c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:47.940988 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d2f1be-b330-4c4c-ba87-2bd9ed0825c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:47.941731 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d2f1be-b330-4c4c-ba87-2bd9ed0825c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:47.943300 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d2f1be-b330-4c4c-ba87-2bd9ed0825c3 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:47.955049 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1398/2814] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:53:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:53:47.959026 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d2f1be-b330-4c4c-ba87-2bd9ed0825c3 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:47.970825 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d2f1be-b330-4c4c-ba87-2bd9ed0825c3 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:47.976401 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99782f9-9705-45ae-aad0-26a6da0b1c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:47.977173 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99782f9-9705-45ae-aad0-26a6da0b1c2b admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:47.977579 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99782f9-9705-45ae-aad0-26a6da0b1c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:47.977919 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99782f9-9705-45ae-aad0-26a6da0b1c2b admin admin] PATH_INFO: `/v3/projects/8379ed77d1cf49d7813775a02ec5faa7/users/6e0fa24bfc5c4c13a4b5de250bf1f48a/roles/e1f9b167c51c4a388cbc9594db8a465d` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:47.989060 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1417/2815] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:53:47 2022] POST /identity/v3/projects => generated 385 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:53:47.993536 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b2b71a-cbe9-49be-a189-2eb38d411ec9 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:47.996990 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99782f9-9705-45ae-aad0-26a6da0b1c2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8379ed77d1cf49d7813775a02ec5faa7, user_id=6e0fa24bfc5c4c13a4b5de250bf1f48a, role_id=e1f9b167c51c4a388cbc9594db8a465d)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:48.001037 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99782f9-9705-45ae-aad0-26a6da0b1c2b admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:48.064864 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b2b71a-cbe9-49be-a189-2eb38d411ec9 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:48.073005 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc851874-93f3-4e69-b27a-de4b962c6b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:48.081527 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc851874-93f3-4e69-b27a-de4b962c6b03 admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:48.082124 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc851874-93f3-4e69-b27a-de4b962c6b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:48.082517 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc851874-93f3-4e69-b27a-de4b962c6b03 admin admin] PATH_INFO: `/v3/users` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:48.083545 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc851874-93f3-4e69-b27a-de4b962c6b03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:48.084211 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1399/2816] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:53:47 2022] PUT /identity/v3/projects/8379ed77d1cf49d7813775a02ec5faa7/users/6e0fa24bfc5c4c13a4b5de250bf1f48a/roles/e1f9b167c51c4a388cbc9594db8a465d => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:53:48.093233 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc851874-93f3-4e69-b27a-de4b962c6b03 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:48.098609 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff0cc19-080d-4a9f-826c-f9f83a79eded None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:48.099529 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff0cc19-080d-4a9f-826c-f9f83a79eded None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:48.099924 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff0cc19-080d-4a9f-826c-f9f83a79eded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:48.136253 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-5ff0cc19-080d-4a9f-826c-f9f83a79eded None None] MFA Rules not processed for user `6e0fa24bfc5c4c13a4b5de250bf1f48a`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:53:48.137590 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-5ff0cc19-080d-4a9f-826c-f9f83a79eded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:48.211898 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1400/2817] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:53:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:53:48.216975 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff0cc19-080d-4a9f-826c-f9f83a79eded None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:48.221084 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1418/2818] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:53:47 2022] POST /identity/v3/users => generated 293 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:53:48.224927 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc851874-93f3-4e69-b27a-de4b962c6b03 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:48.236582 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc851874-93f3-4e69-b27a-de4b962c6b03 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:48.314576 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff0cc19-080d-4a9f-826c-f9f83a79eded None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:48.324244 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c071a84f-fd1e-4579-9a69-b08047be216d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:48.325650 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c071a84f-fd1e-4579-9a69-b08047be216d admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:48.326237 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c071a84f-fd1e-4579-9a69-b08047be216d admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:48.326695 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ddfb35-fc8f-4a6f-bab7-d1efb0be86c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:48.327013 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c071a84f-fd1e-4579-9a69-b08047be216d admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:48.327554 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ddfb35-fc8f-4a6f-bab7-d1efb0be86c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:48.327818 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ddfb35-fc8f-4a6f-bab7-d1efb0be86c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:48.328088 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ddfb35-fc8f-4a6f-bab7-d1efb0be86c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:48.328763 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ddfb35-fc8f-4a6f-bab7-d1efb0be86c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:48.329365 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c071a84f-fd1e-4579-9a69-b08047be216d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:48.331010 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ddfb35-fc8f-4a6f-bab7-d1efb0be86c5 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:48.332816 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c071a84f-fd1e-4579-9a69-b08047be216d admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:48.343474 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1419/2819] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:53:48 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 06:53:48.351777 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ddfb35-fc8f-4a6f-bab7-d1efb0be86c5 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:48.371318 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ddfb35-fc8f-4a6f-bab7-d1efb0be86c5 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:48.378906 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2d9d06-b1fb-47cc-be40-6d383e0c402b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:48.379641 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2d9d06-b1fb-47cc-be40-6d383e0c402b admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:48.380209 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2d9d06-b1fb-47cc-be40-6d383e0c402b admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:48.381229 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2d9d06-b1fb-47cc-be40-6d383e0c402b admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:48.381838 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2d9d06-b1fb-47cc-be40-6d383e0c402b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:48.384633 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2d9d06-b1fb-47cc-be40-6d383e0c402b admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:48.432485 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1420/2820] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:53:48 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:53:48.435165 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2d9d06-b1fb-47cc-be40-6d383e0c402b admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:48.446780 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2d9d06-b1fb-47cc-be40-6d383e0c402b admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:48.453068 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5a5002-90eb-4743-b463-0b7215df90c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:48.457215 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5a5002-90eb-4743-b463-0b7215df90c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:48.457215 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5a5002-90eb-4743-b463-0b7215df90c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:48.457215 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5a5002-90eb-4743-b463-0b7215df90c7 admin admin] PATH_INFO: `/v3/projects/ed264fd6b9bb4780a0b9057001404782/users/c9b31804a9d942868e16a815fe4a7de7/roles/e1f9b167c51c4a388cbc9594db8a465d` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:48.475777 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5a5002-90eb-4743-b463-0b7215df90c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed264fd6b9bb4780a0b9057001404782, user_id=c9b31804a9d942868e16a815fe4a7de7, role_id=e1f9b167c51c4a388cbc9594db8a465d)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:48.478530 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5a5002-90eb-4743-b463-0b7215df90c7 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:48.572332 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1401/2821] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:53:48 2022] POST /identity/v3/projects => generated 375 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:53:48.576290 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c071a84f-fd1e-4579-9a69-b08047be216d admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:48.605491 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1421/2822] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:53:48 2022] PUT /identity/v3/projects/ed264fd6b9bb4780a0b9057001404782/users/c9b31804a9d942868e16a815fe4a7de7/roles/e1f9b167c51c4a388cbc9594db8a465d => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:53:48.607246 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a11691-532a-45dc-ab8d-137cf4814c90 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:48.607543 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a11691-532a-45dc-ab8d-137cf4814c90 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:48.607824 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a11691-532a-45dc-ab8d-137cf4814c90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:48.662867 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-85a11691-532a-45dc-ab8d-137cf4814c90 None None] MFA Rules not processed for user `c9b31804a9d942868e16a815fe4a7de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:53:48.664434 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-85a11691-532a-45dc-ab8d-137cf4814c90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:48.715229 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c071a84f-fd1e-4579-9a69-b08047be216d admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:48.777260 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1422/2823] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:53:48 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:53:48.779629 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a11691-532a-45dc-ab8d-137cf4814c90 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:48.849158 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b33078-0864-4464-bc3e-64baebcafa1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:48.850176 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b33078-0864-4464-bc3e-64baebcafa1b admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:48.850176 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b33078-0864-4464-bc3e-64baebcafa1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:48.850411 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b33078-0864-4464-bc3e-64baebcafa1b admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:48.851116 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b33078-0864-4464-bc3e-64baebcafa1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:48.853558 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b33078-0864-4464-bc3e-64baebcafa1b admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:48.893067 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a11691-532a-45dc-ab8d-137cf4814c90 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:48.898473 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a511e741-85a2-4007-a692-02cbc90c6dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:48.899533 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a511e741-85a2-4007-a692-02cbc90c6dd6 admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:48.899970 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a511e741-85a2-4007-a692-02cbc90c6dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:48.900409 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a511e741-85a2-4007-a692-02cbc90c6dd6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:48.901227 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a511e741-85a2-4007-a692-02cbc90c6dd6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:48.903530 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a511e741-85a2-4007-a692-02cbc90c6dd6 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:48.953100 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:53:48.958611 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1402/2824] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:53:48 2022] POST /identity/v3/users => generated 294 bytes in 380 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:53:48.958611 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b33078-0864-4464-bc3e-64baebcafa1b admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:48.960285 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-84b33078-0864-4464-bc3e-64baebcafa1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:49.043133 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b33078-0864-4464-bc3e-64baebcafa1b admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:49.053229 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46521062-848c-4b64-b9a2-1d29b417cd5f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '306188d42dbd4b5b84082ab3842eaf9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:49.059052 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1423/2825] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:53:48 2022] POST /identity/v3/projects => generated 387 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:53:49.059052 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a511e741-85a2-4007-a692-02cbc90c6dd6 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:49.059255 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46521062-848c-4b64-b9a2-1d29b417cd5f service placement] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:49.059255 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46521062-848c-4b64-b9a2-1d29b417cd5f service placement] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:49.059599 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46521062-848c-4b64-b9a2-1d29b417cd5f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:49.063354 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-46521062-848c-4b64-b9a2-1d29b417cd5f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:49.079764 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46521062-848c-4b64-b9a2-1d29b417cd5f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:49.082267 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46521062-848c-4b64-b9a2-1d29b417cd5f service placement] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:49.091014 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:53:49.102328 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1403/2826] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:53:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:53:49.103286 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46521062-848c-4b64-b9a2-1d29b417cd5f service placement] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:49.115453 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46521062-848c-4b64-b9a2-1d29b417cd5f service placement] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:49.168357 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a511e741-85a2-4007-a692-02cbc90c6dd6 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:49.175105 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563f7074-0a45-4649-8a83-a59b12efb997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:49.176468 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563f7074-0a45-4649-8a83-a59b12efb997 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:49.176468 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563f7074-0a45-4649-8a83-a59b12efb997 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:49.176692 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563f7074-0a45-4649-8a83-a59b12efb997 admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:49.177721 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563f7074-0a45-4649-8a83-a59b12efb997 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:49.179368 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563f7074-0a45-4649-8a83-a59b12efb997 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:49.198075 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1424/2827] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:53:49 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:53:49.199328 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563f7074-0a45-4649-8a83-a59b12efb997 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:49.209963 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae289e9-ea46-42cd-8e16-0054d0bd5096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:49.210540 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563f7074-0a45-4649-8a83-a59b12efb997 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:49.211231 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae289e9-ea46-42cd-8e16-0054d0bd5096 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:49.211578 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae289e9-ea46-42cd-8e16-0054d0bd5096 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:49.211796 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae289e9-ea46-42cd-8e16-0054d0bd5096 admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:49.213267 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae289e9-ea46-42cd-8e16-0054d0bd5096 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:49.215305 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae289e9-ea46-42cd-8e16-0054d0bd5096 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:49.223625 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1efc13-9bdc-47c8-93bd-932525877769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:49.224317 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1efc13-9bdc-47c8-93bd-932525877769 admin admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:49.224598 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1efc13-9bdc-47c8-93bd-932525877769 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:49.224866 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1efc13-9bdc-47c8-93bd-932525877769 admin admin] PATH_INFO: `/v3/projects/73834b7fdb4a4f66a70911b548a291bc/users/314235f9596e4f1da4f6907ce69e4094/roles/486144ad1bcd417e86acdbc1d14327bf` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:49.254635 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1efc13-9bdc-47c8-93bd-932525877769 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73834b7fdb4a4f66a70911b548a291bc, user_id=314235f9596e4f1da4f6907ce69e4094, role_id=486144ad1bcd417e86acdbc1d14327bf)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:49.254635 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1efc13-9bdc-47c8-93bd-932525877769 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:49.387764 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1425/2828] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:53:49 2022] PUT /identity/v3/projects/73834b7fdb4a4f66a70911b548a291bc/users/314235f9596e4f1da4f6907ce69e4094/roles/486144ad1bcd417e86acdbc1d14327bf => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:53:49.394124 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1efc13-9bdc-47c8-93bd-932525877769 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:49.411234 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1404/2829] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:53:49 2022] POST /identity/v3/users => generated 300 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:53:49.416633 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae289e9-ea46-42cd-8e16-0054d0bd5096 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:49.450424 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae289e9-ea46-42cd-8e16-0054d0bd5096 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:49.534929 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1efc13-9bdc-47c8-93bd-932525877769 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:49.546849 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e941f0-a7b0-48c4-9d60-b61186d3dd92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:49.547502 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e941f0-a7b0-48c4-9d60-b61186d3dd92 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:49.547782 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e941f0-a7b0-48c4-9d60-b61186d3dd92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:49.548530 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e941f0-a7b0-48c4-9d60-b61186d3dd92 admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:49.548708 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e941f0-a7b0-48c4-9d60-b61186d3dd92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:49.550381 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e941f0-a7b0-48c4-9d60-b61186d3dd92 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:49.576258 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1426/2830] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:53:49 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:53:49.583234 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e941f0-a7b0-48c4-9d60-b61186d3dd92 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:49.585866 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:53:49.603915 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e941f0-a7b0-48c4-9d60-b61186d3dd92 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:49.616118 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2b3ead-af48-4c3d-96b4-2a2cdf9d3a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:49.616840 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2b3ead-af48-4c3d-96b4-2a2cdf9d3a7b admin admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:49.617066 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2b3ead-af48-4c3d-96b4-2a2cdf9d3a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:49.617369 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2b3ead-af48-4c3d-96b4-2a2cdf9d3a7b admin admin] PATH_INFO: `/v3/projects/73834b7fdb4a4f66a70911b548a291bc/users/314235f9596e4f1da4f6907ce69e4094/roles/486144ad1bcd417e86acdbc1d14327bf` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:49.626741 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2b3ead-af48-4c3d-96b4-2a2cdf9d3a7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=73834b7fdb4a4f66a70911b548a291bc, user_id=314235f9596e4f1da4f6907ce69e4094, role_id=486144ad1bcd417e86acdbc1d14327bf)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:49.629480 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2b3ead-af48-4c3d-96b4-2a2cdf9d3a7b admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:49.639523 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:53:49.643905 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f8d8d3-cb0d-4d77-9744-2e83be074704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:49.644647 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f8d8d3-cb0d-4d77-9744-2e83be074704 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:49.645107 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f8d8d3-cb0d-4d77-9744-2e83be074704 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:49.647557 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f8d8d3-cb0d-4d77-9744-2e83be074704 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:49.647557 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f8d8d3-cb0d-4d77-9744-2e83be074704 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:49.651335 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f8d8d3-cb0d-4d77-9744-2e83be074704 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:49.654165 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1427/2831] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:53:49 2022] PUT /identity/v3/projects/73834b7fdb4a4f66a70911b548a291bc/users/314235f9596e4f1da4f6907ce69e4094/roles/486144ad1bcd417e86acdbc1d14327bf => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:53:49.669292 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1405/2832] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:53:49 2022] GET /identity/v3/roles => generated 1420 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:53:49.674061 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2b3ead-af48-4c3d-96b4-2a2cdf9d3a7b admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:49.744586 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f788c10-8439-4922-9ea4-8281b83c5482 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:49.745070 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f788c10-8439-4922-9ea4-8281b83c5482 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:49.745396 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f788c10-8439-4922-9ea4-8281b83c5482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:49.791618 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2b3ead-af48-4c3d-96b4-2a2cdf9d3a7b admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:49.793720 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-7f788c10-8439-4922-9ea4-8281b83c5482 None None] MFA Rules not processed for user `314235f9596e4f1da4f6907ce69e4094`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:53:49.795097 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-7f788c10-8439-4922-9ea4-8281b83c5482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:49.817519 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3b99b9-6405-481a-bc92-0b629f48751b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:49.824812 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3b99b9-6405-481a-bc92-0b629f48751b admin admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:49.825385 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3b99b9-6405-481a-bc92-0b629f48751b admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:49.825673 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3b99b9-6405-481a-bc92-0b629f48751b admin admin] PATH_INFO: `/v3/projects/4c0d92b7cbc742b6b946655f3f006e70/users/7b7b90e89ce94f5ea8b3b325c4ad8a50/roles/486144ad1bcd417e86acdbc1d14327bf` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:49.877995 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3b99b9-6405-481a-bc92-0b629f48751b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c0d92b7cbc742b6b946655f3f006e70, user_id=7b7b90e89ce94f5ea8b3b325c4ad8a50, role_id=486144ad1bcd417e86acdbc1d14327bf)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:49.881170 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3b99b9-6405-481a-bc92-0b629f48751b admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:50.002245 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1428/2833] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:53:49 2022] PUT /identity/v3/projects/4c0d92b7cbc742b6b946655f3f006e70/users/7b7b90e89ce94f5ea8b3b325c4ad8a50/roles/486144ad1bcd417e86acdbc1d14327bf => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:53:50.003321 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1406/2834] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:53:49 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:53:50.004288 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3b99b9-6405-481a-bc92-0b629f48751b admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:50.014002 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f788c10-8439-4922-9ea4-8281b83c5482 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:50.016678 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-7f788c10-8439-4922-9ea4-8281b83c5482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:50.139073 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f788c10-8439-4922-9ea4-8281b83c5482 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:50.146430 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3b99b9-6405-481a-bc92-0b629f48751b admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:50.152852 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687f9515-9a13-4596-a9b5-ab94b7eb65c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:50.153510 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687f9515-9a13-4596-a9b5-ab94b7eb65c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:50.153798 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687f9515-9a13-4596-a9b5-ab94b7eb65c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:50.154071 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687f9515-9a13-4596-a9b5-ab94b7eb65c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:50.155710 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687f9515-9a13-4596-a9b5-ab94b7eb65c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:50.157590 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687f9515-9a13-4596-a9b5-ab94b7eb65c2 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:50.171947 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1429/2835] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:53:50 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:53:50.175982 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687f9515-9a13-4596-a9b5-ab94b7eb65c2 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:50.178666 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:53:50.191846 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687f9515-9a13-4596-a9b5-ab94b7eb65c2 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:50.198198 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10765132-f7d6-4a3c-9d1d-032f566d4683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:50.199097 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10765132-f7d6-4a3c-9d1d-032f566d4683 admin admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:50.199596 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10765132-f7d6-4a3c-9d1d-032f566d4683 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:50.200031 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10765132-f7d6-4a3c-9d1d-032f566d4683 admin admin] PATH_INFO: `/v3/projects/4c0d92b7cbc742b6b946655f3f006e70/users/7b7b90e89ce94f5ea8b3b325c4ad8a50/roles/486144ad1bcd417e86acdbc1d14327bf` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:50.203913 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10765132-f7d6-4a3c-9d1d-032f566d4683 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c0d92b7cbc742b6b946655f3f006e70, user_id=7b7b90e89ce94f5ea8b3b325c4ad8a50, role_id=486144ad1bcd417e86acdbc1d14327bf)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:50.208112 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10765132-f7d6-4a3c-9d1d-032f566d4683 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:50.225294 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1430/2836] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:53:50 2022] PUT /identity/v3/projects/4c0d92b7cbc742b6b946655f3f006e70/users/7b7b90e89ce94f5ea8b3b325c4ad8a50/roles/486144ad1bcd417e86acdbc1d14327bf => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:53:50.236578 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46996b1b-a39d-4a00-909d-5d0e85303823 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:50.236859 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46996b1b-a39d-4a00-909d-5d0e85303823 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:50.237511 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46996b1b-a39d-4a00-909d-5d0e85303823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:50.246837 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:53:50.253835 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b49d397-d02b-4bad-a0cb-f364c07dcae8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36639dce4f97412f89d71367f650b901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:50.254457 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b49d397-d02b-4bad-a0cb-f364c07dcae8 service cinder] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:50.254726 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b49d397-d02b-4bad-a0cb-f364c07dcae8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:50.255010 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b49d397-d02b-4bad-a0cb-f364c07dcae8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:50.290629 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-46996b1b-a39d-4a00-909d-5d0e85303823 None None] MFA Rules not processed for user `7b7b90e89ce94f5ea8b3b325c4ad8a50`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:53:50.292111 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-46996b1b-a39d-4a00-909d-5d0e85303823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:50.378129 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b49d397-d02b-4bad-a0cb-f364c07dcae8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:50.382324 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b49d397-d02b-4bad-a0cb-f364c07dcae8 service cinder] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:50.400380 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1407/2837] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:53:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:53:50.426148 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1431/2838] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:53:50 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:53:50.427233 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b49d397-d02b-4bad-a0cb-f364c07dcae8 service cinder] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:50.428760 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-0b49d397-d02b-4bad-a0cb-f364c07dcae8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:50.441912 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b49d397-d02b-4bad-a0cb-f364c07dcae8 service cinder] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:50.534833 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17106624-5d3e-4dc4-a458-b5727ac41e73 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36639dce4f97412f89d71367f650b901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:50.536736 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17106624-5d3e-4dc4-a458-b5727ac41e73 service cinder] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:50.537051 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17106624-5d3e-4dc4-a458-b5727ac41e73 service cinder] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:50.537388 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17106624-5d3e-4dc4-a458-b5727ac41e73 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:50.550769 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17106624-5d3e-4dc4-a458-b5727ac41e73 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:50.555969 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17106624-5d3e-4dc4-a458-b5727ac41e73 service cinder] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:50.572119 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1408/2839] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:53:50 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:53:51.927826 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46996b1b-a39d-4a00-909d-5d0e85303823 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:51.996213 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46996b1b-a39d-4a00-909d-5d0e85303823 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:52.001478 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2329dc71-7264-44b7-882d-36157425c9de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:52.002101 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2329dc71-7264-44b7-882d-36157425c9de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:52.002387 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2329dc71-7264-44b7-882d-36157425c9de admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:52.002663 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2329dc71-7264-44b7-882d-36157425c9de admin admin] PATH_INFO: `/v3/users/c9b31804a9d942868e16a815fe4a7de7` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:52.003955 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2329dc71-7264-44b7-882d-36157425c9de admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9b31804a9d942868e16a815fe4a7de7)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:52.006016 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2329dc71-7264-44b7-882d-36157425c9de admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:52.033652 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17106624-5d3e-4dc4-a458-b5727ac41e73 service cinder] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:52.035152 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-17106624-5d3e-4dc4-a458-b5727ac41e73 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:52.051060 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17106624-5d3e-4dc4-a458-b5727ac41e73 service cinder] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:52.055944 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17e025a-44f7-4860-89bf-aae22c218de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:52.056610 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17e025a-44f7-4860-89bf-aae22c218de5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:52.056876 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17e025a-44f7-4860-89bf-aae22c218de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:52.057149 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17e025a-44f7-4860-89bf-aae22c218de5 admin admin] PATH_INFO: `/v3/users/594e9324c44540568a81fd5fdb980efb` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:52.071688 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17e025a-44f7-4860-89bf-aae22c218de5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=594e9324c44540568a81fd5fdb980efb)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:52.073633 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17e025a-44f7-4860-89bf-aae22c218de5 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:52.304395 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-2329dc71-7264-44b7-882d-36157425c9de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9b31804a9d942868e16a815fe4a7de7'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:52.304395 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-2329dc71-7264-44b7-882d-36157425c9de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9b31804a9d942868e16a815fe4a7de7'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:52.304395 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-2329dc71-7264-44b7-882d-36157425c9de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9b31804a9d942868e16a815fe4a7de7'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:52.354750 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-c17e025a-44f7-4860-89bf-aae22c218de5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '594e9324c44540568a81fd5fdb980efb'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:52.355442 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-c17e025a-44f7-4860-89bf-aae22c218de5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '594e9324c44540568a81fd5fdb980efb'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:52.356118 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-c17e025a-44f7-4860-89bf-aae22c218de5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '594e9324c44540568a81fd5fdb980efb'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:52.364084 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-c17e025a-44f7-4860-89bf-aae22c218de5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '594e9324c44540568a81fd5fdb980efb'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:52.368548 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:53:52.369166 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-2329dc71-7264-44b7-882d-36157425c9de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9b31804a9d942868e16a815fe4a7de7'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:52.382758 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-2329dc71-7264-44b7-882d-36157425c9de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9b31804a9d942868e16a815fe4a7de7'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:52.395673 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1432/2840] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:53:51 2022] DELETE /identity/v3/users/c9b31804a9d942868e16a815fe4a7de7 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:53:52.400062 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c684b7-a208-464e-8779-586989745aa6 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:52.400351 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c684b7-a208-464e-8779-586989745aa6 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:52.400637 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c684b7-a208-464e-8779-586989745aa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:52.414272 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:53:52.420311 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-e4c684b7-a208-464e-8779-586989745aa6 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:53:52.422057 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-e4c684b7-a208-464e-8779-586989745aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:52.449481 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-c17e025a-44f7-4860-89bf-aae22c218de5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '594e9324c44540568a81fd5fdb980efb'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:52.469395 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1409/2841] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:53:52 2022] DELETE /identity/v3/users/594e9324c44540568a81fd5fdb980efb => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:53:52.484923 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1433/2842] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:53:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:53:52.494817 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17e025a-44f7-4860-89bf-aae22c218de5 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:52.496301 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-c17e025a-44f7-4860-89bf-aae22c218de5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:52.571743 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17e025a-44f7-4860-89bf-aae22c218de5 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:52.582061 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e90d00-d272-4dea-8e1f-ef70f19a4c7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:52.582811 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e90d00-d272-4dea-8e1f-ef70f19a4c7b service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:52.583099 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e90d00-d272-4dea-8e1f-ef70f19a4c7b service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:52.583370 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e90d00-d272-4dea-8e1f-ef70f19a4c7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:52.662604 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e90d00-d272-4dea-8e1f-ef70f19a4c7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:52.664104 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e90d00-d272-4dea-8e1f-ef70f19a4c7b service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:52.675589 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1410/2843] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:53:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:53:52.735617 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c684b7-a208-464e-8779-586989745aa6 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:52.738047 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-e4c684b7-a208-464e-8779-586989745aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:52.756656 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c684b7-a208-464e-8779-586989745aa6 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:52.762485 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70d0673-3d6e-442b-8eef-88203e02926b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:52.763334 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70d0673-3d6e-442b-8eef-88203e02926b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:52.763957 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70d0673-3d6e-442b-8eef-88203e02926b admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:52.765129 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70d0673-3d6e-442b-8eef-88203e02926b admin admin] PATH_INFO: `/v3/projects/a97277d100304d6aa689f73174562474` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:52.778711 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70d0673-3d6e-442b-8eef-88203e02926b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a97277d100304d6aa689f73174562474)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:52.781244 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70d0673-3d6e-442b-8eef-88203e02926b admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:52.905230 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-b70d0673-3d6e-442b-8eef-88203e02926b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a97277d100304d6aa689f73174562474', 'request_id': 'req-b70d0673-3d6e-442b-8eef-88203e02926b'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:52.980120 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-b70d0673-3d6e-442b-8eef-88203e02926b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a97277d100304d6aa689f73174562474', 'request_id': 'req-b70d0673-3d6e-442b-8eef-88203e02926b'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:52.990664 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-b70d0673-3d6e-442b-8eef-88203e02926b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a97277d100304d6aa689f73174562474', 'request_id': 'req-b70d0673-3d6e-442b-8eef-88203e02926b'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:53.003544 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-b70d0673-3d6e-442b-8eef-88203e02926b admin admin] The token cache is being invalidate because project a97277d100304d6aa689f73174562474 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81723) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:53:53.003913 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-b70d0673-3d6e-442b-8eef-88203e02926b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:53.004845 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-b70d0673-3d6e-442b-8eef-88203e02926b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:53.006651 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1434/2844] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:53:52 2022] DELETE /identity/v3/projects/a97277d100304d6aa689f73174562474 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:53:53.010543 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e90d00-d272-4dea-8e1f-ef70f19a4c7b service neutron] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:53.021107 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e90d00-d272-4dea-8e1f-ef70f19a4c7b service neutron] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:53.088872 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d4d023-8158-41a1-9f1e-dcaeb0b7de49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:53.089615 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d4d023-8158-41a1-9f1e-dcaeb0b7de49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:53.089810 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d4d023-8158-41a1-9f1e-dcaeb0b7de49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:53.090082 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d4d023-8158-41a1-9f1e-dcaeb0b7de49 admin admin] PATH_INFO: `/v3/users/d138aba8c4654fd0a3e08ea068315492` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:53.103988 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d4d023-8158-41a1-9f1e-dcaeb0b7de49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d138aba8c4654fd0a3e08ea068315492)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:53.105991 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d4d023-8158-41a1-9f1e-dcaeb0b7de49 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:53.130032 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:53:53.177124 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70d0673-3d6e-442b-8eef-88203e02926b admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:53.178737 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-b70d0673-3d6e-442b-8eef-88203e02926b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:53.192550 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70d0673-3d6e-442b-8eef-88203e02926b admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:53.197666 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0874ef-280d-4e36-b6bc-0e1e447a54ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:53.198301 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0874ef-280d-4e36-b6bc-0e1e447a54ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:53.198582 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0874ef-280d-4e36-b6bc-0e1e447a54ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:53.198854 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0874ef-280d-4e36-b6bc-0e1e447a54ab admin admin] PATH_INFO: `/v3/projects/ed264fd6b9bb4780a0b9057001404782` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:53.200086 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0874ef-280d-4e36-b6bc-0e1e447a54ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed264fd6b9bb4780a0b9057001404782)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:53.209420 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0874ef-280d-4e36-b6bc-0e1e447a54ab admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:53.337068 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-a7d4d023-8158-41a1-9f1e-dcaeb0b7de49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd138aba8c4654fd0a3e08ea068315492'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:53.337717 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-a7d4d023-8158-41a1-9f1e-dcaeb0b7de49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd138aba8c4654fd0a3e08ea068315492'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:53.338331 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-a7d4d023-8158-41a1-9f1e-dcaeb0b7de49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd138aba8c4654fd0a3e08ea068315492'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:53.338726 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-6d0874ef-280d-4e36-b6bc-0e1e447a54ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed264fd6b9bb4780a0b9057001404782', 'request_id': 'req-6d0874ef-280d-4e36-b6bc-0e1e447a54ab'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:53.346531 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-a7d4d023-8158-41a1-9f1e-dcaeb0b7de49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd138aba8c4654fd0a3e08ea068315492'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:53.397294 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-6d0874ef-280d-4e36-b6bc-0e1e447a54ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed264fd6b9bb4780a0b9057001404782', 'request_id': 'req-6d0874ef-280d-4e36-b6bc-0e1e447a54ab'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:53.403999 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-6d0874ef-280d-4e36-b6bc-0e1e447a54ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed264fd6b9bb4780a0b9057001404782', 'request_id': 'req-6d0874ef-280d-4e36-b6bc-0e1e447a54ab'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:53.415651 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-6d0874ef-280d-4e36-b6bc-0e1e447a54ab admin admin] The token cache is being invalidate because project ed264fd6b9bb4780a0b9057001404782 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81723) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:53:53.417023 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-6d0874ef-280d-4e36-b6bc-0e1e447a54ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:53.417023 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-6d0874ef-280d-4e36-b6bc-0e1e447a54ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:53.418219 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1435/2845] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:53:53 2022] DELETE /identity/v3/projects/ed264fd6b9bb4780a0b9057001404782 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:53:53.422943 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0874ef-280d-4e36-b6bc-0e1e447a54ab admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:53.424341 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-6d0874ef-280d-4e36-b6bc-0e1e447a54ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:53.430626 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:53:53.466050 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-a7d4d023-8158-41a1-9f1e-dcaeb0b7de49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd138aba8c4654fd0a3e08ea068315492'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:53.485792 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1411/2846] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:53:53 2022] DELETE /identity/v3/users/d138aba8c4654fd0a3e08ea068315492 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:53:53.492719 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0874ef-280d-4e36-b6bc-0e1e447a54ab admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:53.551187 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d585ec-a696-49fe-915d-598a57124d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:53.554729 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d585ec-a696-49fe-915d-598a57124d00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:53.554729 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d585ec-a696-49fe-915d-598a57124d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:53.554729 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d585ec-a696-49fe-915d-598a57124d00 admin admin] PATH_INFO: `/v3/users/7b7b90e89ce94f5ea8b3b325c4ad8a50` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:53.554729 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d585ec-a696-49fe-915d-598a57124d00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b7b90e89ce94f5ea8b3b325c4ad8a50)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:53.555538 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d585ec-a696-49fe-915d-598a57124d00 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:53.608849 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d4d023-8158-41a1-9f1e-dcaeb0b7de49 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:53.610334 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-a7d4d023-8158-41a1-9f1e-dcaeb0b7de49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:53.623247 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d4d023-8158-41a1-9f1e-dcaeb0b7de49 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:53.628188 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eccae67-8deb-446e-9a49-5d764189d833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:53.628825 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eccae67-8deb-446e-9a49-5d764189d833 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:53.629102 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eccae67-8deb-446e-9a49-5d764189d833 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:53.629394 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eccae67-8deb-446e-9a49-5d764189d833 admin admin] PATH_INFO: `/v3/users/6e0fa24bfc5c4c13a4b5de250bf1f48a` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:53.630508 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eccae67-8deb-446e-9a49-5d764189d833 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e0fa24bfc5c4c13a4b5de250bf1f48a)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:53.632414 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eccae67-8deb-446e-9a49-5d764189d833 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:53.784651 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-09d585ec-a696-49fe-915d-598a57124d00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b7b90e89ce94f5ea8b3b325c4ad8a50'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:53.785369 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-09d585ec-a696-49fe-915d-598a57124d00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b7b90e89ce94f5ea8b3b325c4ad8a50'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:53.787371 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-09d585ec-a696-49fe-915d-598a57124d00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b7b90e89ce94f5ea8b3b325c4ad8a50'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:53.828476 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-9eccae67-8deb-446e-9a49-5d764189d833 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e0fa24bfc5c4c13a4b5de250bf1f48a'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:53.829232 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-9eccae67-8deb-446e-9a49-5d764189d833 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e0fa24bfc5c4c13a4b5de250bf1f48a'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:53.829920 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-9eccae67-8deb-446e-9a49-5d764189d833 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e0fa24bfc5c4c13a4b5de250bf1f48a'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:53.837361 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-9eccae67-8deb-446e-9a49-5d764189d833 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e0fa24bfc5c4c13a4b5de250bf1f48a'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:53.837750 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-09d585ec-a696-49fe-915d-598a57124d00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b7b90e89ce94f5ea8b3b325c4ad8a50'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:53.851942 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-09d585ec-a696-49fe-915d-598a57124d00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b7b90e89ce94f5ea8b3b325c4ad8a50'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:53.865022 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1436/2847] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:53:53 2022] DELETE /identity/v3/users/7b7b90e89ce94f5ea8b3b325c4ad8a50 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:53:53.865862 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d585ec-a696-49fe-915d-598a57124d00 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:53.867199 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-09d585ec-a696-49fe-915d-598a57124d00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:53.895234 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-9eccae67-8deb-446e-9a49-5d764189d833 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e0fa24bfc5c4c13a4b5de250bf1f48a'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:53.903625 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:53:53.924379 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1412/2848] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:53:53 2022] DELETE /identity/v3/users/6e0fa24bfc5c4c13a4b5de250bf1f48a => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:53:53.928866 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4769dc-9a6a-4a08-910e-80064e44ddd9 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:53.929154 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4769dc-9a6a-4a08-910e-80064e44ddd9 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:53.929475 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4769dc-9a6a-4a08-910e-80064e44ddd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:53.940360 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d585ec-a696-49fe-915d-598a57124d00 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:53.950452 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-7c4769dc-9a6a-4a08-910e-80064e44ddd9 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:53:53.952384 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-7c4769dc-9a6a-4a08-910e-80064e44ddd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:54.004076 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42579f86-69d9-4e99-b41a-82b1919bf505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:54.005005 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42579f86-69d9-4e99-b41a-82b1919bf505 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:54.005559 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42579f86-69d9-4e99-b41a-82b1919bf505 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:54.005837 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42579f86-69d9-4e99-b41a-82b1919bf505 admin admin] PATH_INFO: `/v3/projects/6148a6eba0e64d2bbec03dcec1b225e9` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:54.016561 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1413/2849] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:53:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:53:54.020421 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42579f86-69d9-4e99-b41a-82b1919bf505 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6148a6eba0e64d2bbec03dcec1b225e9)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:54.023164 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42579f86-69d9-4e99-b41a-82b1919bf505 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:54.026912 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4769dc-9a6a-4a08-910e-80064e44ddd9 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:54.028324 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-7c4769dc-9a6a-4a08-910e-80064e44ddd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:54.039905 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:53:54.092804 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4769dc-9a6a-4a08-910e-80064e44ddd9 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:54.097429 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43cb1b1-30d9-489e-b797-3bd921bf0177 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:54.103302 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43cb1b1-30d9-489e-b797-3bd921bf0177 service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:54.103302 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43cb1b1-30d9-489e-b797-3bd921bf0177 service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:54.103302 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43cb1b1-30d9-489e-b797-3bd921bf0177 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:54.111600 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43cb1b1-30d9-489e-b797-3bd921bf0177 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:54.113101 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43cb1b1-30d9-489e-b797-3bd921bf0177 service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:54.124938 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1414/2850] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:53:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:53:54.143853 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-42579f86-69d9-4e99-b41a-82b1919bf505 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6148a6eba0e64d2bbec03dcec1b225e9', 'request_id': 'req-42579f86-69d9-4e99-b41a-82b1919bf505'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:54.226623 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-42579f86-69d9-4e99-b41a-82b1919bf505 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6148a6eba0e64d2bbec03dcec1b225e9', 'request_id': 'req-42579f86-69d9-4e99-b41a-82b1919bf505'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:54.234382 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-42579f86-69d9-4e99-b41a-82b1919bf505 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6148a6eba0e64d2bbec03dcec1b225e9', 'request_id': 'req-42579f86-69d9-4e99-b41a-82b1919bf505'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:54.246069 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-42579f86-69d9-4e99-b41a-82b1919bf505 admin admin] The token cache is being invalidate because project 6148a6eba0e64d2bbec03dcec1b225e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81723) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:53:54.246395 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-42579f86-69d9-4e99-b41a-82b1919bf505 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:54.247032 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-42579f86-69d9-4e99-b41a-82b1919bf505 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:54.249439 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1437/2851] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:53:53 2022] DELETE /identity/v3/projects/6148a6eba0e64d2bbec03dcec1b225e9 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:53:54.419149 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43cb1b1-30d9-489e-b797-3bd921bf0177 service neutron] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:54.421594 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-a43cb1b1-30d9-489e-b797-3bd921bf0177 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:54.435977 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43cb1b1-30d9-489e-b797-3bd921bf0177 service neutron] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:54.535307 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5697946e-a6b2-47ba-a2fb-9b1fccc8b2f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:54.536034 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5697946e-a6b2-47ba-a2fb-9b1fccc8b2f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:54.536311 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5697946e-a6b2-47ba-a2fb-9b1fccc8b2f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:54.536582 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5697946e-a6b2-47ba-a2fb-9b1fccc8b2f7 admin admin] PATH_INFO: `/v3/projects/4c0d92b7cbc742b6b946655f3f006e70` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:54.537748 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5697946e-a6b2-47ba-a2fb-9b1fccc8b2f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c0d92b7cbc742b6b946655f3f006e70)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:54.539635 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5697946e-a6b2-47ba-a2fb-9b1fccc8b2f7 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:54.650129 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-5697946e-a6b2-47ba-a2fb-9b1fccc8b2f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c0d92b7cbc742b6b946655f3f006e70', 'request_id': 'req-5697946e-a6b2-47ba-a2fb-9b1fccc8b2f7'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:54.657107 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-5697946e-a6b2-47ba-a2fb-9b1fccc8b2f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c0d92b7cbc742b6b946655f3f006e70', 'request_id': 'req-5697946e-a6b2-47ba-a2fb-9b1fccc8b2f7'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:54.668404 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42579f86-69d9-4e99-b41a-82b1919bf505 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:54.670174 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-42579f86-69d9-4e99-b41a-82b1919bf505 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:54.683656 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42579f86-69d9-4e99-b41a-82b1919bf505 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:54.744017 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ca8fbd-f25c-4b0a-8dee-5c2da3e61bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:54.744641 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca8fbd-f25c-4b0a-8dee-5c2da3e61bc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:54.744926 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca8fbd-f25c-4b0a-8dee-5c2da3e61bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:54.745223 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca8fbd-f25c-4b0a-8dee-5c2da3e61bc9 admin admin] PATH_INFO: `/v3/projects/8379ed77d1cf49d7813775a02ec5faa7` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:54.746414 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ca8fbd-f25c-4b0a-8dee-5c2da3e61bc9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8379ed77d1cf49d7813775a02ec5faa7)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:54.748329 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ca8fbd-f25c-4b0a-8dee-5c2da3e61bc9 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:54.750088 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:53:54.751497 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-5697946e-a6b2-47ba-a2fb-9b1fccc8b2f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c0d92b7cbc742b6b946655f3f006e70', 'request_id': 'req-5697946e-a6b2-47ba-a2fb-9b1fccc8b2f7'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:54.765008 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-5697946e-a6b2-47ba-a2fb-9b1fccc8b2f7 admin admin] The token cache is being invalidate because project 4c0d92b7cbc742b6b946655f3f006e70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:53:54.765339 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-5697946e-a6b2-47ba-a2fb-9b1fccc8b2f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:54.765994 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-5697946e-a6b2-47ba-a2fb-9b1fccc8b2f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:54.767476 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1415/2852] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:53:54 2022] DELETE /identity/v3/projects/4c0d92b7cbc742b6b946655f3f006e70 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:53:54.775296 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905492e6-6b60-45cb-8224-2037cd247b48 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:54.775296 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905492e6-6b60-45cb-8224-2037cd247b48 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:54.775296 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905492e6-6b60-45cb-8224-2037cd247b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:54.793705 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-905492e6-6b60-45cb-8224-2037cd247b48 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:53:54.794899 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-905492e6-6b60-45cb-8224-2037cd247b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:54.803720 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1416/2853] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:53:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:53:54.875935 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f1f48-1f6e-4950-a326-6a00c1cacb6a None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:54.876313 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f1f48-1f6e-4950-a326-6a00c1cacb6a None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:54.876592 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f1f48-1f6e-4950-a326-6a00c1cacb6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:54.884562 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-b5ca8fbd-f25c-4b0a-8dee-5c2da3e61bc9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8379ed77d1cf49d7813775a02ec5faa7', 'request_id': 'req-b5ca8fbd-f25c-4b0a-8dee-5c2da3e61bc9'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:54.898098 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-b99f1f48-1f6e-4950-a326-6a00c1cacb6a None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:53:54.899292 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-b99f1f48-1f6e-4950-a326-6a00c1cacb6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:54.930726 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:53:54.960154 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1417/2854] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:53:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:53:54.963495 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-b5ca8fbd-f25c-4b0a-8dee-5c2da3e61bc9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8379ed77d1cf49d7813775a02ec5faa7', 'request_id': 'req-b5ca8fbd-f25c-4b0a-8dee-5c2da3e61bc9'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:54.964612 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99f1f48-1f6e-4950-a326-6a00c1cacb6a None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:54.971384 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-b5ca8fbd-f25c-4b0a-8dee-5c2da3e61bc9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8379ed77d1cf49d7813775a02ec5faa7', 'request_id': 'req-b5ca8fbd-f25c-4b0a-8dee-5c2da3e61bc9'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:54.974671 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99f1f48-1f6e-4950-a326-6a00c1cacb6a None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:54.979883 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bf3445-1edf-430b-af3c-6c7db2075cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:54.980501 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bf3445-1edf-430b-af3c-6c7db2075cc0 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:54.980807 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bf3445-1edf-430b-af3c-6c7db2075cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:54.981083 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bf3445-1edf-430b-af3c-6c7db2075cc0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:54.981801 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bf3445-1edf-430b-af3c-6c7db2075cc0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:53:54.982106 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bf3445-1edf-430b-af3c-6c7db2075cc0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:54.984075 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-b5ca8fbd-f25c-4b0a-8dee-5c2da3e61bc9 admin admin] The token cache is being invalidate because project 8379ed77d1cf49d7813775a02ec5faa7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81723) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:53:54.984400 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-b5ca8fbd-f25c-4b0a-8dee-5c2da3e61bc9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:54.984643 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bf3445-1edf-430b-af3c-6c7db2075cc0 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:54.985058 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-b5ca8fbd-f25c-4b0a-8dee-5c2da3e61bc9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:54.986581 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1438/2855] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:53:54 2022] DELETE /identity/v3/projects/8379ed77d1cf49d7813775a02ec5faa7 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:53:54.990993 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ca8fbd-f25c-4b0a-8dee-5c2da3e61bc9 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:54.992841 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-b5ca8fbd-f25c-4b0a-8dee-5c2da3e61bc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:54.996742 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1418/2856] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 24 06:53:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 06:53:55.001128 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bf3445-1edf-430b-af3c-6c7db2075cc0 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:55.003390 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-b9bf3445-1edf-430b-af3c-6c7db2075cc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:55.018062 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ca8fbd-f25c-4b0a-8dee-5c2da3e61bc9 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:55.018062 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bd6048-9f01-4de6-b1a1-4e526347e85c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:55.018062 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bd6048-9f01-4de6-b1a1-4e526347e85c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:55.018062 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bd6048-9f01-4de6-b1a1-4e526347e85c admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:55.018892 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bd6048-9f01-4de6-b1a1-4e526347e85c admin admin] PATH_INFO: `/v3/users/314235f9596e4f1da4f6907ce69e4094` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:55.018892 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bd6048-9f01-4de6-b1a1-4e526347e85c admin admin] RBAC: Authorizing `identity:delete_user(user_id=314235f9596e4f1da4f6907ce69e4094)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:55.019077 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bf3445-1edf-430b-af3c-6c7db2075cc0 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:55.019171 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bd6048-9f01-4de6-b1a1-4e526347e85c admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:55.021208 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e66bcac-92bc-40b7-87dc-aed254737202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:55.021816 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e66bcac-92bc-40b7-87dc-aed254737202 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:55.022097 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e66bcac-92bc-40b7-87dc-aed254737202 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:55.022368 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e66bcac-92bc-40b7-87dc-aed254737202 admin admin] PATH_INFO: `/v3/projects` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:55.022993 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e66bcac-92bc-40b7-87dc-aed254737202 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:55.024998 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e66bcac-92bc-40b7-87dc-aed254737202 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:55.149397 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1419/2857] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:53:55 2022] POST /identity/v3/projects => generated 375 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:53:55.154374 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e66bcac-92bc-40b7-87dc-aed254737202 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:55.216245 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e66bcac-92bc-40b7-87dc-aed254737202 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:55.222308 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b710c9d-f0d5-4b04-9579-67578dc77c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:55.222920 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b710c9d-f0d5-4b04-9579-67578dc77c94 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:55.223199 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b710c9d-f0d5-4b04-9579-67578dc77c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:55.223473 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b710c9d-f0d5-4b04-9579-67578dc77c94 admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:55.224093 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b710c9d-f0d5-4b04-9579-67578dc77c94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:55.226089 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b710c9d-f0d5-4b04-9579-67578dc77c94 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:55.262740 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-e7bd6048-9f01-4de6-b1a1-4e526347e85c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '314235f9596e4f1da4f6907ce69e4094'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:55.263397 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-e7bd6048-9f01-4de6-b1a1-4e526347e85c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '314235f9596e4f1da4f6907ce69e4094'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:55.263993 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-e7bd6048-9f01-4de6-b1a1-4e526347e85c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '314235f9596e4f1da4f6907ce69e4094'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:55.351456 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-e7bd6048-9f01-4de6-b1a1-4e526347e85c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '314235f9596e4f1da4f6907ce69e4094'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:55.360799 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1420/2858] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:53:55 2022] POST /identity/v3/users => generated 288 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:53:55.361942 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b710c9d-f0d5-4b04-9579-67578dc77c94 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:55.363741 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-4b710c9d-f0d5-4b04-9579-67578dc77c94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:55.364977 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-e7bd6048-9f01-4de6-b1a1-4e526347e85c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '314235f9596e4f1da4f6907ce69e4094'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:55.375980 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b710c9d-f0d5-4b04-9579-67578dc77c94 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:55.379160 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1439/2859] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:53:54 2022] DELETE /identity/v3/users/314235f9596e4f1da4f6907ce69e4094 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:53:55.384121 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b26687c-4051-4d25-8aa9-fbc185cb7f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:55.384779 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b26687c-4051-4d25-8aa9-fbc185cb7f95 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:55.385061 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b26687c-4051-4d25-8aa9-fbc185cb7f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:55.385369 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b26687c-4051-4d25-8aa9-fbc185cb7f95 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:55.386024 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b26687c-4051-4d25-8aa9-fbc185cb7f95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:55.387645 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b26687c-4051-4d25-8aa9-fbc185cb7f95 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:55.398976 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1421/2860] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:53:55 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:53:55.403084 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bd6048-9f01-4de6-b1a1-4e526347e85c admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:55.471882 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bd6048-9f01-4de6-b1a1-4e526347e85c admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:55.476898 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7922c79-cf04-4bda-9cb7-be03afec239f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:55.477709 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7922c79-cf04-4bda-9cb7-be03afec239f admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:55.477835 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7922c79-cf04-4bda-9cb7-be03afec239f admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:55.478058 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7922c79-cf04-4bda-9cb7-be03afec239f admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:55.478817 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7922c79-cf04-4bda-9cb7-be03afec239f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:55.480553 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7922c79-cf04-4bda-9cb7-be03afec239f admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:55.483099 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:53:55.491354 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1440/2861] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:53:55 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:53:55.495731 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b26687c-4051-4d25-8aa9-fbc185cb7f95 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:55.506474 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b26687c-4051-4d25-8aa9-fbc185cb7f95 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:55.512247 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc64b49-bdc2-4a6d-9a00-9d037b1b056c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:55.512931 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc64b49-bdc2-4a6d-9a00-9d037b1b056c admin admin] REQUEST_METHOD: `PUT` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:55.516364 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc64b49-bdc2-4a6d-9a00-9d037b1b056c admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:55.516364 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc64b49-bdc2-4a6d-9a00-9d037b1b056c admin admin] PATH_INFO: `/v3/projects/7f668670ad564f9aab5124c167160fc5/users/0daacb4835b44deb921c7344998be162/roles/e1f9b167c51c4a388cbc9594db8a465d` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:55.529219 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc64b49-bdc2-4a6d-9a00-9d037b1b056c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f668670ad564f9aab5124c167160fc5, user_id=0daacb4835b44deb921c7344998be162, role_id=e1f9b167c51c4a388cbc9594db8a465d)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:55.532177 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc64b49-bdc2-4a6d-9a00-9d037b1b056c admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:55.603915 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1422/2862] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:53:55 2022] PUT /identity/v3/projects/7f668670ad564f9aab5124c167160fc5/users/0daacb4835b44deb921c7344998be162/roles/e1f9b167c51c4a388cbc9594db8a465d => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:53:55.617299 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfed927-4321-463b-869b-e9d0604d05a1 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:55.617730 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfed927-4321-463b-869b-e9d0604d05a1 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:55.618112 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfed927-4321-463b-869b-e9d0604d05a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:55.652262 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-edfed927-4321-463b-869b-e9d0604d05a1 None None] MFA Rules not processed for user `0daacb4835b44deb921c7344998be162`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:53:55.653629 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-edfed927-4321-463b-869b-e9d0604d05a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:55.728853 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1441/2863] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:53:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 06:53:55.741355 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc64b49-bdc2-4a6d-9a00-9d037b1b056c admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:55.743048 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-ecc64b49-bdc2-4a6d-9a00-9d037b1b056c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:55.802373 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc64b49-bdc2-4a6d-9a00-9d037b1b056c admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:55.805512 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfed927-4321-463b-869b-e9d0604d05a1 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:55.806794 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99afbbd8-2485-43f9-8e54-865112e6539f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36639dce4f97412f89d71367f650b901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:55.814845 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-edfed927-4321-463b-869b-e9d0604d05a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:55.815415 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99afbbd8-2485-43f9-8e54-865112e6539f service cinder] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:55.815415 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99afbbd8-2485-43f9-8e54-865112e6539f service cinder] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:55.815415 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99afbbd8-2485-43f9-8e54-865112e6539f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:55.820224 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99afbbd8-2485-43f9-8e54-865112e6539f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:55.823651 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99afbbd8-2485-43f9-8e54-865112e6539f service cinder] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:55.829211 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:53:55.841449 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1423/2864] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:53:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:53:55.882658 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfed927-4321-463b-869b-e9d0604d05a1 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:55.887535 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31aa889-6e00-44af-908e-e996f0ae5284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:55.888167 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31aa889-6e00-44af-908e-e996f0ae5284 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:55.888699 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31aa889-6e00-44af-908e-e996f0ae5284 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:55.888826 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31aa889-6e00-44af-908e-e996f0ae5284 admin admin] PATH_INFO: `/v3/projects/73834b7fdb4a4f66a70911b548a291bc` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:55.890568 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31aa889-6e00-44af-908e-e996f0ae5284 admin admin] RBAC: Authorizing `identity:delete_project(project_id=73834b7fdb4a4f66a70911b548a291bc)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:55.892642 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31aa889-6e00-44af-908e-e996f0ae5284 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:55.999744 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-d31aa889-6e00-44af-908e-e996f0ae5284 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73834b7fdb4a4f66a70911b548a291bc', 'request_id': 'req-d31aa889-6e00-44af-908e-e996f0ae5284'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:56.089034 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-d31aa889-6e00-44af-908e-e996f0ae5284 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73834b7fdb4a4f66a70911b548a291bc', 'request_id': 'req-d31aa889-6e00-44af-908e-e996f0ae5284'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:56.096952 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-d31aa889-6e00-44af-908e-e996f0ae5284 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73834b7fdb4a4f66a70911b548a291bc', 'request_id': 'req-d31aa889-6e00-44af-908e-e996f0ae5284'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:56.108815 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-d31aa889-6e00-44af-908e-e996f0ae5284 admin admin] The token cache is being invalidate because project 73834b7fdb4a4f66a70911b548a291bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81723) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:53:56.109104 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-d31aa889-6e00-44af-908e-e996f0ae5284 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:56.109807 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-d31aa889-6e00-44af-908e-e996f0ae5284 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:53:56.112440 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1442/2865] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:53:55 2022] DELETE /identity/v3/projects/73834b7fdb4a4f66a70911b548a291bc => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:53:56.116846 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db04f94-a0da-4cdb-a6e5-06168a048d25 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:56.117130 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db04f94-a0da-4cdb-a6e5-06168a048d25 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:56.117458 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db04f94-a0da-4cdb-a6e5-06168a048d25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:56.137367 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-4db04f94-a0da-4cdb-a6e5-06168a048d25 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:53:56.139790 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-4db04f94-a0da-4cdb-a6e5-06168a048d25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:56.199693 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1424/2866] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:53:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:53:56.208142 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afadfe24-2b40-4b99-b52d-bb1c217d1e4d None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:56.208611 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afadfe24-2b40-4b99-b52d-bb1c217d1e4d None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:56.208762 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afadfe24-2b40-4b99-b52d-bb1c217d1e4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:56.229299 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-afadfe24-2b40-4b99-b52d-bb1c217d1e4d None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:53:56.230594 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-afadfe24-2b40-4b99-b52d-bb1c217d1e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:56.238763 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1443/2867] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:53:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:53:56.244650 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db04f94-a0da-4cdb-a6e5-06168a048d25 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:56.256871 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db04f94-a0da-4cdb-a6e5-06168a048d25 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:56.262017 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d581a5-fb75-4403-ad56-188d0a3105c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:56.262623 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d581a5-fb75-4403-ad56-188d0a3105c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:56.262901 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d581a5-fb75-4403-ad56-188d0a3105c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:56.263202 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d581a5-fb75-4403-ad56-188d0a3105c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:56.264017 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d581a5-fb75-4403-ad56-188d0a3105c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:53:56.265325 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d581a5-fb75-4403-ad56-188d0a3105c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:56.266917 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d581a5-fb75-4403-ad56-188d0a3105c4 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:56.278856 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1425/2868] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 24 06:53:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:53:56.283941 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afadfe24-2b40-4b99-b52d-bb1c217d1e4d None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:56.293877 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afadfe24-2b40-4b99-b52d-bb1c217d1e4d None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:56.298954 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6a4323-34b7-4b73-b543-05fec0bd2deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:56.299579 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6a4323-34b7-4b73-b543-05fec0bd2deb admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:56.299853 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6a4323-34b7-4b73-b543-05fec0bd2deb admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:56.300119 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6a4323-34b7-4b73-b543-05fec0bd2deb admin admin] PATH_INFO: `/v3/projects` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:56.300733 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6a4323-34b7-4b73-b543-05fec0bd2deb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:56.302657 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6a4323-34b7-4b73-b543-05fec0bd2deb admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:56.314179 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:53:56.433357 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1444/2869] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:53:56 2022] POST /identity/v3/projects => generated 367 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:53:56.438238 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d581a5-fb75-4403-ad56-188d0a3105c4 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:56.448876 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d581a5-fb75-4403-ad56-188d0a3105c4 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:56.507033 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f0983c-6d81-499e-a68f-a80ebe2b81c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:56.507522 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f0983c-6d81-499e-a68f-a80ebe2b81c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:56.507797 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f0983c-6d81-499e-a68f-a80ebe2b81c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:56.508200 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f0983c-6d81-499e-a68f-a80ebe2b81c3 admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:56.508848 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f0983c-6d81-499e-a68f-a80ebe2b81c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:56.510832 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f0983c-6d81-499e-a68f-a80ebe2b81c3 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:56.592534 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1426/2870] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:53:56 2022] POST /identity/v3/users => generated 284 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 06:53:56.597710 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6a4323-34b7-4b73-b543-05fec0bd2deb admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:56.609112 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6a4323-34b7-4b73-b543-05fec0bd2deb admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:56.614630 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4217f1c3-207c-42e1-a5b0-df98eb7ea16f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:56.615243 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4217f1c3-207c-42e1-a5b0-df98eb7ea16f admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:56.615526 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4217f1c3-207c-42e1-a5b0-df98eb7ea16f admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:56.615802 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4217f1c3-207c-42e1-a5b0-df98eb7ea16f admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:56.617040 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4217f1c3-207c-42e1-a5b0-df98eb7ea16f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:56.622242 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4217f1c3-207c-42e1-a5b0-df98eb7ea16f admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:56.630482 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1445/2871] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:53:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:53:56.634420 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f0983c-6d81-499e-a68f-a80ebe2b81c3 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:56.645249 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f0983c-6d81-499e-a68f-a80ebe2b81c3 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:56.650321 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06cee49-96de-4f01-90ab-d06221c300c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:56.651008 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06cee49-96de-4f01-90ab-d06221c300c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:56.651356 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06cee49-96de-4f01-90ab-d06221c300c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:56.651697 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06cee49-96de-4f01-90ab-d06221c300c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:56.652410 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06cee49-96de-4f01-90ab-d06221c300c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:56.654123 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06cee49-96de-4f01-90ab-d06221c300c4 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:56.666299 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1427/2872] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:53:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:53:56.671006 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4217f1c3-207c-42e1-a5b0-df98eb7ea16f admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:56.681747 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4217f1c3-207c-42e1-a5b0-df98eb7ea16f admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:56.687536 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a2b323-7a4a-492a-b578-c0408a9674bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:56.688306 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a2b323-7a4a-492a-b578-c0408a9674bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:56.688589 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a2b323-7a4a-492a-b578-c0408a9674bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:56.689401 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a2b323-7a4a-492a-b578-c0408a9674bc admin admin] PATH_INFO: `/v3/projects/e9972954a60240e4be829f9490098f13/users/ca8976a548d847d898ba8733780e638a/roles/e1f9b167c51c4a388cbc9594db8a465d` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:56.704582 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a2b323-7a4a-492a-b578-c0408a9674bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9972954a60240e4be829f9490098f13, user_id=ca8976a548d847d898ba8733780e638a, role_id=e1f9b167c51c4a388cbc9594db8a465d)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:56.707307 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a2b323-7a4a-492a-b578-c0408a9674bc admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:56.832216 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1446/2873] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:53:56 2022] PUT /identity/v3/projects/e9972954a60240e4be829f9490098f13/users/ca8976a548d847d898ba8733780e638a/roles/e1f9b167c51c4a388cbc9594db8a465d => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:53:56.842737 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bee7dc-4fb6-403b-80e3-b992ef0f3176 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:56.843018 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bee7dc-4fb6-403b-80e3-b992ef0f3176 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:56.843312 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bee7dc-4fb6-403b-80e3-b992ef0f3176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:56.872311 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-90bee7dc-4fb6-403b-80e3-b992ef0f3176 None None] MFA Rules not processed for user `ca8976a548d847d898ba8733780e638a`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:53:56.874284 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-90bee7dc-4fb6-403b-80e3-b992ef0f3176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:56.930029 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:53:56.940033 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1428/2874] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:53:56 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:53:56.952782 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a2b323-7a4a-492a-b578-c0408a9674bc admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:53:56.954492 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-e8a2b323-7a4a-492a-b578-c0408a9674bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:53:57.038973 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a2b323-7a4a-492a-b578-c0408a9674bc admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:53:57.042703 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e1365e-5bbd-4d88-9956-d2278252bda1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b331b6ddfe46aeae7b52fcbbc27771', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:53:57.043385 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e1365e-5bbd-4d88-9956-d2278252bda1 service glance] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:53:57.043676 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e1365e-5bbd-4d88-9956-d2278252bda1 service glance] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:53:57.043953 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e1365e-5bbd-4d88-9956-d2278252bda1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:53:57.053592 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e1365e-5bbd-4d88-9956-d2278252bda1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:53:57.055038 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e1365e-5bbd-4d88-9956-d2278252bda1 service glance] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:53:57.064136 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1447/2875] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:53:56 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:54:07.036991 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=3 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:54:07.059962 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=54,keystone:INSERT=2 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:54:29.390236 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bee7dc-4fb6-403b-80e3-b992ef0f3176 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:54:29.391651 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-90bee7dc-4fb6-403b-80e3-b992ef0f3176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:54:29.459383 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bee7dc-4fb6-403b-80e3-b992ef0f3176 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:54:29.464174 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8ba4e1-f8dc-4bbe-bfd4-6fddd38065f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:54:29.464808 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8ba4e1-f8dc-4bbe-bfd4-6fddd38065f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:54:29.465089 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8ba4e1-f8dc-4bbe-bfd4-6fddd38065f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:54:29.465397 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8ba4e1-f8dc-4bbe-bfd4-6fddd38065f5 admin admin] PATH_INFO: `/v3/users/0daacb4835b44deb921c7344998be162` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:54:29.466524 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8ba4e1-f8dc-4bbe-bfd4-6fddd38065f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0daacb4835b44deb921c7344998be162)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:54:29.468493 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8ba4e1-f8dc-4bbe-bfd4-6fddd38065f5 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:54:29.718691 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-bc8ba4e1-f8dc-4bbe-bfd4-6fddd38065f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0daacb4835b44deb921c7344998be162'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:54:29.719372 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-bc8ba4e1-f8dc-4bbe-bfd4-6fddd38065f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0daacb4835b44deb921c7344998be162'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:54:29.719976 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-bc8ba4e1-f8dc-4bbe-bfd4-6fddd38065f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0daacb4835b44deb921c7344998be162'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:54:29.727927 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-bc8ba4e1-f8dc-4bbe-bfd4-6fddd38065f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0daacb4835b44deb921c7344998be162'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:54:29.819594 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-bc8ba4e1-f8dc-4bbe-bfd4-6fddd38065f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0daacb4835b44deb921c7344998be162'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:54:29.838821 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1429/2876] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:54:29 2022] DELETE /identity/v3/users/0daacb4835b44deb921c7344998be162 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:54:29.843546 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8cde3d-d4cb-4794-89e7-ea5e3f402451 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:54:29.844056 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8cde3d-d4cb-4794-89e7-ea5e3f402451 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:54:29.844174 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8cde3d-d4cb-4794-89e7-ea5e3f402451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:54:29.863550 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-ab8cde3d-d4cb-4794-89e7-ea5e3f402451 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:54:29.865538 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-ab8cde3d-d4cb-4794-89e7-ea5e3f402451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:54:29.927446 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1448/2877] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:54:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:54:29.939463 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8ba4e1-f8dc-4bbe-bfd4-6fddd38065f5 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:54:29.939463 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-bc8ba4e1-f8dc-4bbe-bfd4-6fddd38065f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:54:29.997145 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8ba4e1-f8dc-4bbe-bfd4-6fddd38065f5 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:54:30.000699 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed94772-e9c5-41b6-ba1a-7ee28168da53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:54:30.001405 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed94772-e9c5-41b6-ba1a-7ee28168da53 service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:54:30.001688 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed94772-e9c5-41b6-ba1a-7ee28168da53 service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:54:30.001960 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed94772-e9c5-41b6-ba1a-7ee28168da53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:54:30.011863 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed94772-e9c5-41b6-ba1a-7ee28168da53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:54:30.013328 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed94772-e9c5-41b6-ba1a-7ee28168da53 service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:54:30.023992 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1430/2878] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:54:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:54:30.438088 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8cde3d-d4cb-4794-89e7-ea5e3f402451 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:54:30.440630 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-ab8cde3d-d4cb-4794-89e7-ea5e3f402451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:54:30.452830 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8cde3d-d4cb-4794-89e7-ea5e3f402451 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:54:30.459383 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8edd3dc-a8e9-4fea-9149-e77db71d54cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:54:30.460152 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8edd3dc-a8e9-4fea-9149-e77db71d54cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:54:30.460788 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8edd3dc-a8e9-4fea-9149-e77db71d54cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:54:30.461160 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8edd3dc-a8e9-4fea-9149-e77db71d54cc admin admin] PATH_INFO: `/v3/projects/7f668670ad564f9aab5124c167160fc5` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:54:30.462487 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8edd3dc-a8e9-4fea-9149-e77db71d54cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f668670ad564f9aab5124c167160fc5)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:54:30.464415 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8edd3dc-a8e9-4fea-9149-e77db71d54cc admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:54:30.580528 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-c8edd3dc-a8e9-4fea-9149-e77db71d54cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f668670ad564f9aab5124c167160fc5', 'request_id': 'req-c8edd3dc-a8e9-4fea-9149-e77db71d54cc'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:54:30.670448 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-c8edd3dc-a8e9-4fea-9149-e77db71d54cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f668670ad564f9aab5124c167160fc5', 'request_id': 'req-c8edd3dc-a8e9-4fea-9149-e77db71d54cc'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:54:30.677605 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-c8edd3dc-a8e9-4fea-9149-e77db71d54cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f668670ad564f9aab5124c167160fc5', 'request_id': 'req-c8edd3dc-a8e9-4fea-9149-e77db71d54cc'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:54:30.688425 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-c8edd3dc-a8e9-4fea-9149-e77db71d54cc admin admin] The token cache is being invalidate because project 7f668670ad564f9aab5124c167160fc5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81723) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:54:30.688655 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-c8edd3dc-a8e9-4fea-9149-e77db71d54cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:54:30.689356 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-c8edd3dc-a8e9-4fea-9149-e77db71d54cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:54:30.690865 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1449/2879] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:54:30 2022] DELETE /identity/v3/projects/7f668670ad564f9aab5124c167160fc5 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:54:30.696542 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822ff9ac-d490-41f5-a329-294541d1ec3f None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:54:30.697030 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822ff9ac-d490-41f5-a329-294541d1ec3f None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:54:30.697133 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822ff9ac-d490-41f5-a329-294541d1ec3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:54:30.717072 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-822ff9ac-d490-41f5-a329-294541d1ec3f None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:54:30.718298 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-822ff9ac-d490-41f5-a329-294541d1ec3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:54:30.734940 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:54:30.776981 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1431/2880] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:54:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:54:30.786339 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb323a3a-02ba-45fb-b0d1-3abcb60eb5f9 None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:54:30.786628 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb323a3a-02ba-45fb-b0d1-3abcb60eb5f9 None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:54:30.786910 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb323a3a-02ba-45fb-b0d1-3abcb60eb5f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:54:30.806753 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-cb323a3a-02ba-45fb-b0d1-3abcb60eb5f9 None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:54:30.807951 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-cb323a3a-02ba-45fb-b0d1-3abcb60eb5f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:54:30.815923 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1450/2881] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:54:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:54:30.821055 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822ff9ac-d490-41f5-a329-294541d1ec3f None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:54:30.830923 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822ff9ac-d490-41f5-a329-294541d1ec3f None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:54:30.836530 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eabe6d5-e26e-46ca-8e2c-6fb0db17dba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:54:30.837128 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eabe6d5-e26e-46ca-8e2c-6fb0db17dba1 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:54:30.837437 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eabe6d5-e26e-46ca-8e2c-6fb0db17dba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:54:30.837708 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eabe6d5-e26e-46ca-8e2c-6fb0db17dba1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:54:30.838392 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eabe6d5-e26e-46ca-8e2c-6fb0db17dba1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81724) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 06:54:30.838692 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eabe6d5-e26e-46ca-8e2c-6fb0db17dba1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:54:30.840265 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eabe6d5-e26e-46ca-8e2c-6fb0db17dba1 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:54:30.852213 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1432/2882] 10.222.0.47 () {58 vars in 1153 bytes} [Wed Aug 24 06:54:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 06:54:30.856814 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb323a3a-02ba-45fb-b0d1-3abcb60eb5f9 None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:54:30.866921 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb323a3a-02ba-45fb-b0d1-3abcb60eb5f9 None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:54:30.872007 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e071f70-335b-49d9-9eba-8bfd6d313224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:54:30.872607 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e071f70-335b-49d9-9eba-8bfd6d313224 admin admin] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:54:30.872886 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e071f70-335b-49d9-9eba-8bfd6d313224 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:54:30.873161 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e071f70-335b-49d9-9eba-8bfd6d313224 admin admin] PATH_INFO: `/v3/projects` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:54:30.873809 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e071f70-335b-49d9-9eba-8bfd6d313224 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:54:30.875703 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e071f70-335b-49d9-9eba-8bfd6d313224 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:54:30.964730 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1451/2883] 10.222.0.47 () {60 vars in 1153 bytes} [Wed Aug 24 06:54:30 2022] POST /identity/v3/projects => generated 383 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:54:30.969037 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eabe6d5-e26e-46ca-8e2c-6fb0db17dba1 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:54:30.979179 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eabe6d5-e26e-46ca-8e2c-6fb0db17dba1 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:54:31.037022 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b24607-0667-457b-801b-b9bd73d39ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:54:31.037795 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b24607-0667-457b-801b-b9bd73d39ce3 admin admin] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:54:31.038078 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b24607-0667-457b-801b-b9bd73d39ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:54:31.038355 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b24607-0667-457b-801b-b9bd73d39ce3 admin admin] PATH_INFO: `/v3/users` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:54:31.038977 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b24607-0667-457b-801b-b9bd73d39ce3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:54:31.040906 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b24607-0667-457b-801b-b9bd73d39ce3 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:54:31.162689 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1433/2884] 10.222.0.47 () {60 vars in 1144 bytes} [Wed Aug 24 06:54:30 2022] POST /identity/v3/users => generated 292 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 06:54:31.167177 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e071f70-335b-49d9-9eba-8bfd6d313224 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:54:31.178501 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e071f70-335b-49d9-9eba-8bfd6d313224 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:54:31.183185 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1490a85-88a5-4aad-a490-bd7f3ffa84b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:54:31.183783 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1490a85-88a5-4aad-a490-bd7f3ffa84b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:54:31.184067 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1490a85-88a5-4aad-a490-bd7f3ffa84b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:54:31.184366 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1490a85-88a5-4aad-a490-bd7f3ffa84b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:54:31.185012 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1490a85-88a5-4aad-a490-bd7f3ffa84b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:54:31.186641 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1490a85-88a5-4aad-a490-bd7f3ffa84b3 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:54:31.197680 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1452/2885] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:54:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:54:31.201686 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b24607-0667-457b-801b-b9bd73d39ce3 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:54:31.211971 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b24607-0667-457b-801b-b9bd73d39ce3 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:54:31.217476 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8b3d95-422f-4b3d-a410-4b7338ad89f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:54:31.217998 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b3d95-422f-4b3d-a410-4b7338ad89f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:54:31.218279 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b3d95-422f-4b3d-a410-4b7338ad89f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:54:31.218550 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b3d95-422f-4b3d-a410-4b7338ad89f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:54:31.219189 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8b3d95-422f-4b3d-a410-4b7338ad89f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:54:31.220768 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8b3d95-422f-4b3d-a410-4b7338ad89f7 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:54:31.231742 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1434/2886] 10.222.0.47 () {58 vars in 1122 bytes} [Wed Aug 24 06:54:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 06:54:31.236059 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1490a85-88a5-4aad-a490-bd7f3ffa84b3 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:54:31.246074 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1490a85-88a5-4aad-a490-bd7f3ffa84b3 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:54:31.250676 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a685e36-4142-43e5-8d61-77693c35e9e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:54:31.251323 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a685e36-4142-43e5-8d61-77693c35e9e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:54:31.251600 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a685e36-4142-43e5-8d61-77693c35e9e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:54:31.251872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a685e36-4142-43e5-8d61-77693c35e9e7 admin admin] PATH_INFO: `/v3/projects/e443552ae9eb446a94665b5674b57f15/users/8bc75750fe3348c18cde49a64456f879/roles/e1f9b167c51c4a388cbc9594db8a465d` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:54:31.268014 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a685e36-4142-43e5-8d61-77693c35e9e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e443552ae9eb446a94665b5674b57f15, user_id=8bc75750fe3348c18cde49a64456f879, role_id=e1f9b167c51c4a388cbc9594db8a465d)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:54:31.271287 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a685e36-4142-43e5-8d61-77693c35e9e7 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:54:31.355966 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1453/2887] 10.222.0.47 () {60 vars in 1483 bytes} [Wed Aug 24 06:54:31 2022] PUT /identity/v3/projects/e443552ae9eb446a94665b5674b57f15/users/8bc75750fe3348c18cde49a64456f879/roles/e1f9b167c51c4a388cbc9594db8a465d => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 06:54:31.367807 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d00f0f8-dbd7-49e6-b145-ca00b75d401f None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:54:31.368089 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d00f0f8-dbd7-49e6-b145-ca00b75d401f None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:54:31.368358 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d00f0f8-dbd7-49e6-b145-ca00b75d401f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:54:31.387885 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-3d00f0f8-dbd7-49e6-b145-ca00b75d401f None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:54:31.389088 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-3d00f0f8-dbd7-49e6-b145-ca00b75d401f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:54:31.447728 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1435/2888] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:54:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:54:31.457449 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a685e36-4142-43e5-8d61-77693c35e9e7 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:54:31.459622 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-1a685e36-4142-43e5-8d61-77693c35e9e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:54:31.481966 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:54:31.518104 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a685e36-4142-43e5-8d61-77693c35e9e7 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:54:31.522055 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8886c18-3dee-43ee-bd24-272c6329ce47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:54:31.522655 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8886c18-3dee-43ee-bd24-272c6329ce47 service neutron] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:54:31.522934 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8886c18-3dee-43ee-bd24-272c6329ce47 service neutron] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:54:31.523228 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8886c18-3dee-43ee-bd24-272c6329ce47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:54:31.532113 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8886c18-3dee-43ee-bd24-272c6329ce47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:54:31.533572 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8886c18-3dee-43ee-bd24-272c6329ce47 service neutron] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:54:31.543332 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1454/2889] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:54:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:54:32.831923 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b03c13-e832-422c-a948-2776d71a9864 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:54:32.834206 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b03c13-e832-422c-a948-2776d71a9864 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:54:32.834206 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b03c13-e832-422c-a948-2776d71a9864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:54:32.862817 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-41b03c13-e832-422c-a948-2776d71a9864 None None] MFA Rules not processed for user `8bc75750fe3348c18cde49a64456f879`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:54:32.865308 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-41b03c13-e832-422c-a948-2776d71a9864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:54:32.881076 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:54:32.933114 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1436/2890] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:54:32 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:54:32.944043 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8886c18-3dee-43ee-bd24-272c6329ce47 service neutron] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:54:32.945763 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-b8886c18-3dee-43ee-bd24-272c6329ce47 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:54:33.003815 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8886c18-3dee-43ee-bd24-272c6329ce47 service neutron] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:54:33.007804 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecb6fe4-becd-4df1-9e1e-7074cc389093 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36639dce4f97412f89d71367f650b901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:54:33.008413 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecb6fe4-becd-4df1-9e1e-7074cc389093 service cinder] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:54:33.008951 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecb6fe4-becd-4df1-9e1e-7074cc389093 service cinder] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:54:33.008951 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecb6fe4-becd-4df1-9e1e-7074cc389093 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:54:33.018887 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecb6fe4-becd-4df1-9e1e-7074cc389093 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:54:33.020919 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecb6fe4-becd-4df1-9e1e-7074cc389093 service cinder] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:54:33.030437 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1455/2891] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:54:32 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:54:35.872194 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16be62ac-ea92-49aa-bd0d-e59bc9c0def4 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:54:35.872792 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16be62ac-ea92-49aa-bd0d-e59bc9c0def4 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:54:35.872792 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16be62ac-ea92-49aa-bd0d-e59bc9c0def4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:54:35.893090 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-16be62ac-ea92-49aa-bd0d-e59bc9c0def4 None None] MFA Rules not processed for user `8bc75750fe3348c18cde49a64456f879`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:54:35.898777 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-16be62ac-ea92-49aa-bd0d-e59bc9c0def4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:54:35.901483 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1437/2892] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:54:35 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:54:36.138996 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecb6fe4-becd-4df1-9e1e-7074cc389093 service cinder] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:54:36.149308 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecb6fe4-becd-4df1-9e1e-7074cc389093 service cinder] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:54:36.152891 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6afcf4-0048-4a94-8aa8-f525535c0497 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:54:36.153581 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6afcf4-0048-4a94-8aa8-f525535c0497 service neutron] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:54:36.153857 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6afcf4-0048-4a94-8aa8-f525535c0497 service neutron] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:54:36.154137 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6afcf4-0048-4a94-8aa8-f525535c0497 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:54:36.156031 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-fe6afcf4-0048-4a94-8aa8-f525535c0497 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:54:36.166726 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6afcf4-0048-4a94-8aa8-f525535c0497 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:54:36.169001 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6afcf4-0048-4a94-8aa8-f525535c0497 service neutron] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:54:36.181256 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1456/2893] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:54:36 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:54:45.887830 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=22 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:54:46.175123 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=49 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:54:52.861830 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16be62ac-ea92-49aa-bd0d-e59bc9c0def4 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:54:52.865042 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-16be62ac-ea92-49aa-bd0d-e59bc9c0def4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:54:52.883194 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16be62ac-ea92-49aa-bd0d-e59bc9c0def4 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:54:52.887874 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72b937c-36a9-45d3-b55d-d7ad4d719bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:54:52.888546 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72b937c-36a9-45d3-b55d-d7ad4d719bc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:54:52.888828 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72b937c-36a9-45d3-b55d-d7ad4d719bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:54:52.889105 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72b937c-36a9-45d3-b55d-d7ad4d719bc0 admin admin] PATH_INFO: `/v3/users/ca8976a548d847d898ba8733780e638a` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:54:52.890306 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72b937c-36a9-45d3-b55d-d7ad4d719bc0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca8976a548d847d898ba8733780e638a)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:54:52.892236 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72b937c-36a9-45d3-b55d-d7ad4d719bc0 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:54:53.095587 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-d72b937c-36a9-45d3-b55d-d7ad4d719bc0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca8976a548d847d898ba8733780e638a'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:54:53.096239 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-d72b937c-36a9-45d3-b55d-d7ad4d719bc0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca8976a548d847d898ba8733780e638a'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:54:53.096843 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-d72b937c-36a9-45d3-b55d-d7ad4d719bc0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca8976a548d847d898ba8733780e638a'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:54:53.104737 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-d72b937c-36a9-45d3-b55d-d7ad4d719bc0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca8976a548d847d898ba8733780e638a'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:54:53.167226 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-d72b937c-36a9-45d3-b55d-d7ad4d719bc0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca8976a548d847d898ba8733780e638a'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:54:53.185348 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1438/2894] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:54:52 2022] DELETE /identity/v3/users/ca8976a548d847d898ba8733780e638a => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:54:53.190044 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a36dd9-aba3-439c-9974-5f2545299e4e None None] REQUEST_METHOD: `POST` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:54:53.190646 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a36dd9-aba3-439c-9974-5f2545299e4e None None] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:54:53.190646 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a36dd9-aba3-439c-9974-5f2545299e4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:54:53.212924 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.auth.core [None req-e7a36dd9-aba3-439c-9974-5f2545299e4e None None] MFA Rules not processed for user `029600e45e6b473db6751098bdba45e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81723) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:54:53.214180 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-e7a36dd9-aba3-439c-9974-5f2545299e4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:54:53.280555 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1457/2895] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:54:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:54:53.291284 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72b937c-36a9-45d3-b55d-d7ad4d719bc0 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:54:53.292691 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-d72b937c-36a9-45d3-b55d-d7ad4d719bc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:54:53.360708 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72b937c-36a9-45d3-b55d-d7ad4d719bc0 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:54:53.365667 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a7f4dd-4790-48e6-8279-be4409aa0391 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:54:53.366917 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a7f4dd-4790-48e6-8279-be4409aa0391 service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:54:53.367204 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a7f4dd-4790-48e6-8279-be4409aa0391 service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:54:53.367479 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a7f4dd-4790-48e6-8279-be4409aa0391 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:54:53.378216 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a7f4dd-4790-48e6-8279-be4409aa0391 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:54:53.379671 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a7f4dd-4790-48e6-8279-be4409aa0391 service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:54:53.390935 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1439/2896] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:54:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:54:53.814844 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a36dd9-aba3-439c-9974-5f2545299e4e None None] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:54:53.816285 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-e7a36dd9-aba3-439c-9974-5f2545299e4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:54:53.833330 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a36dd9-aba3-439c-9974-5f2545299e4e None None] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:54:53.839186 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dd3c9e-9c33-4746-a6bd-a84579c33d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:54:53.840577 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dd3c9e-9c33-4746-a6bd-a84579c33d89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:54:53.841030 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dd3c9e-9c33-4746-a6bd-a84579c33d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:54:53.841367 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dd3c9e-9c33-4746-a6bd-a84579c33d89 admin admin] PATH_INFO: `/v3/projects/e9972954a60240e4be829f9490098f13` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:54:53.842694 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dd3c9e-9c33-4746-a6bd-a84579c33d89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9972954a60240e4be829f9490098f13)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:54:53.844886 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dd3c9e-9c33-4746-a6bd-a84579c33d89 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:54:53.970024 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-21dd3c9e-9c33-4746-a6bd-a84579c33d89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9972954a60240e4be829f9490098f13', 'request_id': 'req-21dd3c9e-9c33-4746-a6bd-a84579c33d89'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:54:54.030240 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-21dd3c9e-9c33-4746-a6bd-a84579c33d89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9972954a60240e4be829f9490098f13', 'request_id': 'req-21dd3c9e-9c33-4746-a6bd-a84579c33d89'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:54:54.037146 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-21dd3c9e-9c33-4746-a6bd-a84579c33d89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9972954a60240e4be829f9490098f13', 'request_id': 'req-21dd3c9e-9c33-4746-a6bd-a84579c33d89'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:54:54.048336 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-21dd3c9e-9c33-4746-a6bd-a84579c33d89 admin admin] The token cache is being invalidate because project e9972954a60240e4be829f9490098f13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81723) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:54:54.048627 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-21dd3c9e-9c33-4746-a6bd-a84579c33d89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:54:54.049402 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-21dd3c9e-9c33-4746-a6bd-a84579c33d89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:54:54.050897 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1458/2897] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:54:53 2022] DELETE /identity/v3/projects/e9972954a60240e4be829f9490098f13 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:55:03.386553 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=55,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:55:04.045374 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:55:46.237673 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a7f4dd-4790-48e6-8279-be4409aa0391 service neutron] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:55:46.239585 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-31a7f4dd-4790-48e6-8279-be4409aa0391 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:55:46.307006 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a7f4dd-4790-48e6-8279-be4409aa0391 service neutron] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:55:46.360081 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6a360e-cb96-40fe-b1ec-7692403cb4d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36639dce4f97412f89d71367f650b901', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:55:46.360758 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6a360e-cb96-40fe-b1ec-7692403cb4d1 service cinder] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:46.361031 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6a360e-cb96-40fe-b1ec-7692403cb4d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:46.361331 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6a360e-cb96-40fe-b1ec-7692403cb4d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:46.363296 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-be6a360e-cb96-40fe-b1ec-7692403cb4d1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:55:46.437921 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6a360e-cb96-40fe-b1ec-7692403cb4d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:55:46.439288 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6a360e-cb96-40fe-b1ec-7692403cb4d1 service cinder] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:55:46.450479 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1440/2898] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:55:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:55:49.520712 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dd3c9e-9c33-4746-a6bd-a84579c33d89 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:55:49.522285 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-21dd3c9e-9c33-4746-a6bd-a84579c33d89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:55:49.591461 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dd3c9e-9c33-4746-a6bd-a84579c33d89 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:55:49.595204 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7895fa-7950-48d7-8fb7-cf12fcc0c8a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:55:49.595806 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7895fa-7950-48d7-8fb7-cf12fcc0c8a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:55:49.596119 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7895fa-7950-48d7-8fb7-cf12fcc0c8a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:55:49.596408 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7895fa-7950-48d7-8fb7-cf12fcc0c8a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:55:49.598185 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-5d7895fa-7950-48d7-8fb7-cf12fcc0c8a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:55:49.612458 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7895fa-7950-48d7-8fb7-cf12fcc0c8a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:55:49.613910 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7895fa-7950-48d7-8fb7-cf12fcc0c8a0 service neutron] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:55:49.625801 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1459/2899] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:55:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:55:56.445413 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=74 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:55:59.619950 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=26 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:56:03.518497 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e469f3e-5ff0-4af2-852f-836320d3dfe4 None None] REQUEST_METHOD: `POST` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:56:03.519092 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e469f3e-5ff0-4af2-852f-836320d3dfe4 None None] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:56:03.519214 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e469f3e-5ff0-4af2-852f-836320d3dfe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:56:03.538957 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.auth.core [None req-3e469f3e-5ff0-4af2-852f-836320d3dfe4 None None] MFA Rules not processed for user `8bc75750fe3348c18cde49a64456f879`. Rule list: `[]` (Enabled: `True`). {{(pid=81724) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 06:56:03.540184 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-3e469f3e-5ff0-4af2-852f-836320d3dfe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:56:03.596924 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1441/2900] 10.222.0.47 () {58 vars in 958 bytes} [Wed Aug 24 06:56:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 06:56:13.589743 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=25 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:56:54.393732 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7895fa-7950-48d7-8fb7-cf12fcc0c8a0 service neutron] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:56:54.402731 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7895fa-7950-48d7-8fb7-cf12fcc0c8a0 service neutron] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:56:54.406586 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cff322-3644-4626-a39d-a45758ac50be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:56:54.407186 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cff322-3644-4626-a39d-a45758ac50be service neutron] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:56:54.407469 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cff322-3644-4626-a39d-a45758ac50be service neutron] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:56:54.407744 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cff322-3644-4626-a39d-a45758ac50be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:56:54.411157 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-c6cff322-3644-4626-a39d-a45758ac50be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:56:54.421607 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cff322-3644-4626-a39d-a45758ac50be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:56:54.423071 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cff322-3644-4626-a39d-a45758ac50be service neutron] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:56:54.432341 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1460/2901] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:56:54 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:56:54.509110 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e469f3e-5ff0-4af2-852f-836320d3dfe4 None None] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:56:54.510617 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-3e469f3e-5ff0-4af2-852f-836320d3dfe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:56:54.523472 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e469f3e-5ff0-4af2-852f-836320d3dfe4 None None] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:56:54.526979 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c008c6-82cd-4944-910e-bc1664c3a339 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:56:54.527584 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c008c6-82cd-4944-910e-bc1664c3a339 service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:56:54.527862 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c008c6-82cd-4944-910e-bc1664c3a339 service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:56:54.528136 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c008c6-82cd-4944-910e-bc1664c3a339 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:56:54.529878 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-05c008c6-82cd-4944-910e-bc1664c3a339 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:56:54.544088 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c008c6-82cd-4944-910e-bc1664c3a339 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:56:54.545561 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c008c6-82cd-4944-910e-bc1664c3a339 service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:56:54.556745 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1442/2902] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:56:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:57:04.428705 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=6 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:57:04.552384 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=6 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:57:16.231363 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cff322-3644-4626-a39d-a45758ac50be service neutron] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:57:16.232849 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-c6cff322-3644-4626-a39d-a45758ac50be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:57:16.305735 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cff322-3644-4626-a39d-a45758ac50be service neutron] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:57:16.310966 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e52f35-cf80-4057-8559-fe2cb3f95693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:57:16.311589 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e52f35-cf80-4057-8559-fe2cb3f95693 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:57:16.311863 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e52f35-cf80-4057-8559-fe2cb3f95693 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:57:16.312134 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e52f35-cf80-4057-8559-fe2cb3f95693 admin admin] PATH_INFO: `/v3/users/8bc75750fe3348c18cde49a64456f879` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:57:16.313327 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e52f35-cf80-4057-8559-fe2cb3f95693 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bc75750fe3348c18cde49a64456f879)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:57:16.315242 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e52f35-cf80-4057-8559-fe2cb3f95693 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:57:16.605871 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-c4e52f35-cf80-4057-8559-fe2cb3f95693 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bc75750fe3348c18cde49a64456f879'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:57:16.606455 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-c4e52f35-cf80-4057-8559-fe2cb3f95693 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bc75750fe3348c18cde49a64456f879'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:57:16.607109 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-c4e52f35-cf80-4057-8559-fe2cb3f95693 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bc75750fe3348c18cde49a64456f879'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:57:16.685107 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-c4e52f35-cf80-4057-8559-fe2cb3f95693 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bc75750fe3348c18cde49a64456f879'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:57:16.697999 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-c4e52f35-cf80-4057-8559-fe2cb3f95693 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bc75750fe3348c18cde49a64456f879'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:57:16.711459 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1461/2903] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:57:16 2022] DELETE /identity/v3/users/8bc75750fe3348c18cde49a64456f879 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:57:16.958534 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c008c6-82cd-4944-910e-bc1664c3a339 service neutron] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:57:16.970431 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c008c6-82cd-4944-910e-bc1664c3a339 service neutron] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:57:17.029170 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6ebf32-697a-4ae6-9336-1950c2c9b508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:57:17.029838 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6ebf32-697a-4ae6-9336-1950c2c9b508 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:57:17.030122 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6ebf32-697a-4ae6-9336-1950c2c9b508 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:57:17.030402 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6ebf32-697a-4ae6-9336-1950c2c9b508 admin admin] PATH_INFO: `/v3/projects/e443552ae9eb446a94665b5674b57f15` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:57:17.031553 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6ebf32-697a-4ae6-9336-1950c2c9b508 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e443552ae9eb446a94665b5674b57f15)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:57:17.033532 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6ebf32-697a-4ae6-9336-1950c2c9b508 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:57:17.163422 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-0c6ebf32-697a-4ae6-9336-1950c2c9b508 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e443552ae9eb446a94665b5674b57f15', 'request_id': 'req-0c6ebf32-697a-4ae6-9336-1950c2c9b508'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:57:17.169719 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-0c6ebf32-697a-4ae6-9336-1950c2c9b508 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e443552ae9eb446a94665b5674b57f15', 'request_id': 'req-0c6ebf32-697a-4ae6-9336-1950c2c9b508'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:57:17.267522 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-0c6ebf32-697a-4ae6-9336-1950c2c9b508 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e443552ae9eb446a94665b5674b57f15', 'request_id': 'req-0c6ebf32-697a-4ae6-9336-1950c2c9b508'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:57:17.279459 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-0c6ebf32-697a-4ae6-9336-1950c2c9b508 admin admin] The token cache is being invalidate because project e443552ae9eb446a94665b5674b57f15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81724) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:57:17.279892 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-0c6ebf32-697a-4ae6-9336-1950c2c9b508 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:57:17.280818 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-0c6ebf32-697a-4ae6-9336-1950c2c9b508 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:57:17.282350 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1443/2904] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:57:16 2022] DELETE /identity/v3/projects/e443552ae9eb446a94665b5674b57f15 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:57:26.707920 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=53,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:57:27.276290 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=51,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:57:47.722794 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e52f35-cf80-4057-8559-fe2cb3f95693 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:57:47.724469 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-c4e52f35-cf80-4057-8559-fe2cb3f95693 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:57:47.785563 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e52f35-cf80-4057-8559-fe2cb3f95693 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:57:47.836135 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900f958b-5f35-49ab-bd12-ec885911d1d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:57:47.836738 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900f958b-5f35-49ab-bd12-ec885911d1d0 service neutron] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:57:47.837015 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900f958b-5f35-49ab-bd12-ec885911d1d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:57:47.837317 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900f958b-5f35-49ab-bd12-ec885911d1d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:57:47.839031 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-900f958b-5f35-49ab-bd12-ec885911d1d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:57:47.849087 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900f958b-5f35-49ab-bd12-ec885911d1d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:57:47.850540 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900f958b-5f35-49ab-bd12-ec885911d1d0 service neutron] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:57:47.858559 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1462/2905] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:57:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:57:57.854872 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=46 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:58:39.945596 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6ebf32-697a-4ae6-9336-1950c2c9b508 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:58:39.947038 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-0c6ebf32-697a-4ae6-9336-1950c2c9b508 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:58:39.960152 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6ebf32-697a-4ae6-9336-1950c2c9b508 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:58:39.963680 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0d502b-4daf-4257-b264-55d4072a54d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:58:39.964271 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0d502b-4daf-4257-b264-55d4072a54d1 service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:58:39.964547 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0d502b-4daf-4257-b264-55d4072a54d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:58:39.964815 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0d502b-4daf-4257-b264-55d4072a54d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:58:39.966506 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-0b0d502b-4daf-4257-b264-55d4072a54d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:58:40.025611 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0d502b-4daf-4257-b264-55d4072a54d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:58:40.027125 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0d502b-4daf-4257-b264-55d4072a54d1 service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:58:40.035984 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1444/2906] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:58:39 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:58:40.942045 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900f958b-5f35-49ab-bd12-ec885911d1d0 service neutron] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:58:40.942045 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-900f958b-5f35-49ab-bd12-ec885911d1d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:58:41.028451 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900f958b-5f35-49ab-bd12-ec885911d1d0 service neutron] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:58:41.034240 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc0f810-2cd2-4bb0-8dfb-64fda66290a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f4f49c1b08c460f8ba6c451c9c92c2a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'ResellerAdmin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:58:41.034848 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc0f810-2cd2-4bb0-8dfb-64fda66290a6 service nova] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:58:41.035139 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc0f810-2cd2-4bb0-8dfb-64fda66290a6 service nova] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:58:41.035425 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc0f810-2cd2-4bb0-8dfb-64fda66290a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:58:41.037759 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-adc0f810-2cd2-4bb0-8dfb-64fda66290a6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:58:41.051232 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc0f810-2cd2-4bb0-8dfb-64fda66290a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:58:41.052836 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc0f810-2cd2-4bb0-8dfb-64fda66290a6 service nova] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:58:41.064888 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1463/2907] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:58:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:58:47.375460 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0d502b-4daf-4257-b264-55d4072a54d1 service neutron] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:58:47.384420 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0d502b-4daf-4257-b264-55d4072a54d1 service neutron] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:58:47.388226 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d8a80c-9d4b-406d-9f6d-f8ff08dc698a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0ba3f03ec15475ab6ebf64994ad742c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:58:47.388836 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d8a80c-9d4b-406d-9f6d-f8ff08dc698a service neutron] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:58:47.389117 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d8a80c-9d4b-406d-9f6d-f8ff08dc698a service neutron] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:58:47.389427 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d8a80c-9d4b-406d-9f6d-f8ff08dc698a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:58:47.391157 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-62d8a80c-9d4b-406d-9f6d-f8ff08dc698a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:58:47.457759 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d8a80c-9d4b-406d-9f6d-f8ff08dc698a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:58:47.459191 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d8a80c-9d4b-406d-9f6d-f8ff08dc698a service neutron] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:58:47.470004 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1445/2908] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:58:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:58:50.798447 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc0f810-2cd2-4bb0-8dfb-64fda66290a6 service nova] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:58:50.800802 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-adc0f810-2cd2-4bb0-8dfb-64fda66290a6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:58:50.954167 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc0f810-2cd2-4bb0-8dfb-64fda66290a6 service nova] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:58:50.959301 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e67b719-1fd6-4cbc-b358-cf6e443445ca service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '306188d42dbd4b5b84082ab3842eaf9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:58:50.959900 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e67b719-1fd6-4cbc-b358-cf6e443445ca service placement] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:58:50.960180 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e67b719-1fd6-4cbc-b358-cf6e443445ca service placement] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:58:50.960457 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e67b719-1fd6-4cbc-b358-cf6e443445ca service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:58:50.962231 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-1e67b719-1fd6-4cbc-b358-cf6e443445ca service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:58:50.974991 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e67b719-1fd6-4cbc-b358-cf6e443445ca service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:58:50.976580 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e67b719-1fd6-4cbc-b358-cf6e443445ca service placement] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:58:50.988297 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1464/2909] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:58:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:58:52.954872 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d8a80c-9d4b-406d-9f6d-f8ff08dc698a service neutron] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:58:53.006563 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d8a80c-9d4b-406d-9f6d-f8ff08dc698a service neutron] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:58:53.012199 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b19985-a5b5-47bf-a66d-e3bb615e5281 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '306188d42dbd4b5b84082ab3842eaf9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:58:53.013441 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b19985-a5b5-47bf-a66d-e3bb615e5281 service placement] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:58:53.013783 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b19985-a5b5-47bf-a66d-e3bb615e5281 service placement] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:58:53.014131 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b19985-a5b5-47bf-a66d-e3bb615e5281 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:58:53.016387 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-60b19985-a5b5-47bf-a66d-e3bb615e5281 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:58:53.030702 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b19985-a5b5-47bf-a66d-e3bb615e5281 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:58:53.032429 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b19985-a5b5-47bf-a66d-e3bb615e5281 service placement] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:58:53.045688 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1446/2910] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:58:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:58:54.540325 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e67b719-1fd6-4cbc-b358-cf6e443445ca service placement] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:58:54.547955 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-1e67b719-1fd6-4cbc-b358-cf6e443445ca service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:58:54.555569 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e67b719-1fd6-4cbc-b358-cf6e443445ca service placement] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:58:54.561853 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171cd61b-62aa-4f06-9b95-fce070710459 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '306188d42dbd4b5b84082ab3842eaf9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:58:54.562477 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171cd61b-62aa-4f06-9b95-fce070710459 service placement] REQUEST_METHOD: `GET` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:58:54.562742 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171cd61b-62aa-4f06-9b95-fce070710459 service placement] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:58:54.563015 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171cd61b-62aa-4f06-9b95-fce070710459 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:58:54.564729 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-171cd61b-62aa-4f06-9b95-fce070710459 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:58:54.572930 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=100 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:58:54.578254 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171cd61b-62aa-4f06-9b95-fce070710459 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:58:54.579863 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171cd61b-62aa-4f06-9b95-fce070710459 service placement] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:58:54.590757 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1465/2911] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 06:58:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:58:54.696483 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b19985-a5b5-47bf-a66d-e3bb615e5281 service placement] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:58:54.696483 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-60b19985-a5b5-47bf-a66d-e3bb615e5281 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:58:54.738245 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b19985-a5b5-47bf-a66d-e3bb615e5281 service placement] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:58:54.742953 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932f7201-1e51-4729-b1fc-59982834f725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:58:54.743585 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932f7201-1e51-4729-b1fc-59982834f725 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:58:54.743870 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932f7201-1e51-4729-b1fc-59982834f725 admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:58:54.744144 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932f7201-1e51-4729-b1fc-59982834f725 admin admin] PATH_INFO: `/v3/users/2e6e7001705b47a09b6e5d7f08595fc8` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:58:54.745306 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932f7201-1e51-4729-b1fc-59982834f725 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e6e7001705b47a09b6e5d7f08595fc8)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:58:54.747309 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932f7201-1e51-4729-b1fc-59982834f725 admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:58:54.761119 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:58:54.952104 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-932f7201-1e51-4729-b1fc-59982834f725 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e6e7001705b47a09b6e5d7f08595fc8'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:58:54.952743 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-932f7201-1e51-4729-b1fc-59982834f725 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e6e7001705b47a09b6e5d7f08595fc8'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:58:54.953411 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-932f7201-1e51-4729-b1fc-59982834f725 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e6e7001705b47a09b6e5d7f08595fc8'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:58:54.960583 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-932f7201-1e51-4729-b1fc-59982834f725 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e6e7001705b47a09b6e5d7f08595fc8'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:58:55.034106 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-932f7201-1e51-4729-b1fc-59982834f725 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e6e7001705b47a09b6e5d7f08595fc8'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:58:55.054480 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1447/2912] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:58:54 2022] DELETE /identity/v3/users/2e6e7001705b47a09b6e5d7f08595fc8 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:58:55.309714 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171cd61b-62aa-4f06-9b95-fce070710459 service placement] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:58:55.321686 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171cd61b-62aa-4f06-9b95-fce070710459 service placement] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:58:55.382329 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f2399a-cb3f-4bc3-aefe-e6949e081c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:58:55.382954 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f2399a-cb3f-4bc3-aefe-e6949e081c16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:58:55.387957 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f2399a-cb3f-4bc3-aefe-e6949e081c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:58:55.387957 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f2399a-cb3f-4bc3-aefe-e6949e081c16 admin admin] PATH_INFO: `/v3/projects/0cc00174433b49a2980339c2963671b0` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:58:55.387957 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f2399a-cb3f-4bc3-aefe-e6949e081c16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0cc00174433b49a2980339c2963671b0)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:58:55.387957 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f2399a-cb3f-4bc3-aefe-e6949e081c16 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:58:55.490677 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-47f2399a-cb3f-4bc3-aefe-e6949e081c16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0cc00174433b49a2980339c2963671b0', 'request_id': 'req-47f2399a-cb3f-4bc3-aefe-e6949e081c16'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:58:55.547808 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-47f2399a-cb3f-4bc3-aefe-e6949e081c16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0cc00174433b49a2980339c2963671b0', 'request_id': 'req-47f2399a-cb3f-4bc3-aefe-e6949e081c16'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:58:55.554718 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-47f2399a-cb3f-4bc3-aefe-e6949e081c16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0cc00174433b49a2980339c2963671b0', 'request_id': 'req-47f2399a-cb3f-4bc3-aefe-e6949e081c16'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:58:55.565363 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-47f2399a-cb3f-4bc3-aefe-e6949e081c16 admin admin] The token cache is being invalidate because project 0cc00174433b49a2980339c2963671b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81723) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:58:55.565697 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-47f2399a-cb3f-4bc3-aefe-e6949e081c16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:58:55.566346 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-47f2399a-cb3f-4bc3-aefe-e6949e081c16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:58:55.567822 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1466/2913] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:58:55 2022] DELETE /identity/v3/projects/0cc00174433b49a2980339c2963671b0 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:58:55.571891 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932f7201-1e51-4729-b1fc-59982834f725 admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:58:55.573311 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-932f7201-1e51-4729-b1fc-59982834f725 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:58:55.586702 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932f7201-1e51-4729-b1fc-59982834f725 admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:58:55.646941 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf408ac3-1892-486c-9c16-f7cc177d604e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:58:55.647586 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf408ac3-1892-486c-9c16-f7cc177d604e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:58:55.647869 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf408ac3-1892-486c-9c16-f7cc177d604e admin admin] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:58:55.648142 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf408ac3-1892-486c-9c16-f7cc177d604e admin admin] PATH_INFO: `/v3/users/c8cd28cddb344330bdd7d58031047b4b` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:58:55.649329 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf408ac3-1892-486c-9c16-f7cc177d604e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8cd28cddb344330bdd7d58031047b4b)` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:58:55.651237 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf408ac3-1892-486c-9c16-f7cc177d604e admin admin] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:58:55.807176 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-cf408ac3-1892-486c-9c16-f7cc177d604e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8cd28cddb344330bdd7d58031047b4b'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:58:55.807845 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-cf408ac3-1892-486c-9c16-f7cc177d604e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8cd28cddb344330bdd7d58031047b4b'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:58:55.808396 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-cf408ac3-1892-486c-9c16-f7cc177d604e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8cd28cddb344330bdd7d58031047b4b'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:58:55.815708 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-cf408ac3-1892-486c-9c16-f7cc177d604e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8cd28cddb344330bdd7d58031047b4b'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:58:55.891128 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.notifications [None req-cf408ac3-1892-486c-9c16-f7cc177d604e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8cd28cddb344330bdd7d58031047b4b'} {{(pid=81724) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:58:55.910080 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1448/2914] 10.222.0.47 () {58 vars in 1224 bytes} [Wed Aug 24 06:58:55 2022] DELETE /identity/v3/users/c8cd28cddb344330bdd7d58031047b4b => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:58:56.172458 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f2399a-cb3f-4bc3-aefe-e6949e081c16 admin admin] Authenticating user token {{(pid=81723) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:58:56.174102 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.fernet_utils [None req-47f2399a-cb3f-4bc3-aefe-e6949e081c16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81723) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:58:56.187922 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f2399a-cb3f-4bc3-aefe-e6949e081c16 admin admin] Validating token access rules against request {{(pid=81723) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:58:56.250586 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffe33de-b398-4c2c-a16b-8c96fbba93b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '029600e45e6b473db6751098bdba45e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24c1e9e70e07496595a602ca047e6a1f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81723) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:58:56.251388 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffe33de-b398-4c2c-a16b-8c96fbba93b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:58:56.251761 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffe33de-b398-4c2c-a16b-8c96fbba93b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:58:56.252031 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffe33de-b398-4c2c-a16b-8c96fbba93b9 admin admin] PATH_INFO: `/v3/projects/0f64ed78d02b462980c827080ae812a9` {{(pid=81723) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:58:56.253415 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffe33de-b398-4c2c-a16b-8c96fbba93b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f64ed78d02b462980c827080ae812a9)` {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:58:56.257049 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffe33de-b398-4c2c-a16b-8c96fbba93b9 admin admin] RBAC: Authorization granted {{(pid=81723) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:58:56.464091 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:58:56.481468 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-fffe33de-b398-4c2c-a16b-8c96fbba93b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f64ed78d02b462980c827080ae812a9', 'request_id': 'req-fffe33de-b398-4c2c-a16b-8c96fbba93b9'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:58:56.570821 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-fffe33de-b398-4c2c-a16b-8c96fbba93b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f64ed78d02b462980c827080ae812a9', 'request_id': 'req-fffe33de-b398-4c2c-a16b-8c96fbba93b9'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:58:56.578477 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-fffe33de-b398-4c2c-a16b-8c96fbba93b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f64ed78d02b462980c827080ae812a9', 'request_id': 'req-fffe33de-b398-4c2c-a16b-8c96fbba93b9'} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:58:56.590312 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-fffe33de-b398-4c2c-a16b-8c96fbba93b9 admin admin] The token cache is being invalidate because project 0f64ed78d02b462980c827080ae812a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81723) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:58:56.595055 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-fffe33de-b398-4c2c-a16b-8c96fbba93b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:58:56.595055 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG keystone.notifications [None req-fffe33de-b398-4c2c-a16b-8c96fbba93b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81723) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:58:56.595055 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: [pid: 81723|app: 0|req: 1467/2915] 10.222.0.47 () {58 vars in 1233 bytes} [Wed Aug 24 06:58:56 2022] DELETE /identity/v3/projects/0f64ed78d02b462980c827080ae812a9 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:59:05.906728 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=74,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:59:06.633977 dv-cinder-854270-1-iscsi devstack@keystone.service[81723]: DEBUG dbcounter [-] [81723] Writing DB stats keystone:DELETE=6,keystone:SELECT=14,keystone:INSERT=1 {{(pid=81723) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 07:03:55.911404 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf408ac3-1892-486c-9c16-f7cc177d604e admin admin] Authenticating user token {{(pid=81724) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 07:03:55.912824 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-cf408ac3-1892-486c-9c16-f7cc177d604e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:03:55.983220 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf408ac3-1892-486c-9c16-f7cc177d604e admin admin] Validating token access rules against request {{(pid=81724) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 07:03:56.042433 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3ba041-38dd-4c18-a1e3-a4d79762ea49 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '306188d42dbd4b5b84082ab3842eaf9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c02e49a89cdd40d885bfa816f06c58a1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81724) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 07:03:56.043089 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3ba041-38dd-4c18-a1e3-a4d79762ea49 service placement] REQUEST_METHOD: `GET` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 07:03:56.043506 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3ba041-38dd-4c18-a1e3-a4d79762ea49 service placement] SCRIPT_NAME: `/identity` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 07:03:56.043847 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3ba041-38dd-4c18-a1e3-a4d79762ea49 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81724) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 07:03:56.046210 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.fernet_utils [None req-3d3ba041-38dd-4c18-a1e3-a4d79762ea49 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81724) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 07:03:56.058373 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3ba041-38dd-4c18-a1e3-a4d79762ea49 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 07:03:56.060544 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3ba041-38dd-4c18-a1e3-a4d79762ea49 service placement] RBAC: Authorization granted {{(pid=81724) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 07:03:56.070652 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: [pid: 81724|app: 0|req: 1449/2916] 10.222.0.47 () {60 vars in 1365 bytes} [Wed Aug 24 07:03:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 07:04:06.066356 dv-cinder-854270-1-iscsi devstack@keystone.service[81724]: DEBUG dbcounter [-] [81724] Writing DB stats keystone:SELECT=58 {{(pid=81724) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}